[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:05 dist-test-slave-g350 krb5kdc[23244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:05 dist-test-slave-g350 krb5kdc[23244](info): set up 2 sockets
Aug 11 10:07:05 dist-test-slave-g350 krb5kdc[23244](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 10:07:06.787953 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:06 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:06.930157 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:33829
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:33829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:07.025036 23257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:07.025308 23257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:07.025339 23257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:07.026738 23257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:07.026789 23257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:07.026808 23257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:07.026823 23257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:07.026836 23257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:07.028926 23257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:33829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:07.029273 23257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:07.029551 23257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:07.037987 23267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.038095 23257 server_base.cc:1034] running on GCE node
W20260811 10:07:07.038362 23265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:07.038995 23264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.039518 23257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:07.040010 23257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:07.041218 23257 hybrid_clock.cc:648] HybridClock initialized: now 1786442827041237 us; error 195 us; skew 500 ppm
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:07.049402 23257 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:07.050220 23257 webserver.cc:460] Webserver started at http://127.22.177.62:35959/ using document root <none> and password file <none>
I20260811 10:07:07.050504 23257 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:07.050557 23257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:07.050770 23257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:07.052217 23257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "e3f4577dea1d4352aac62a0d048d424a"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "133a273a57da404962c7cd4555fda770"
server_key_iv: "34b175e49199bd878028e8606bd42d45"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.052636 23257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "e3f4577dea1d4352aac62a0d048d424a"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "133a273a57da404962c7cd4555fda770"
server_key_iv: "34b175e49199bd878028e8606bd42d45"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.054792 23257 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260811 10:07:07.055994 23274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.056330 23257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:07.056471 23257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e3f4577dea1d4352aac62a0d048d424a"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "133a273a57da404962c7cd4555fda770"
server_key_iv: "34b175e49199bd878028e8606bd42d45"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.056581 23257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:07.069388 23257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:07.072602 23257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:07.072902 23257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:07.084794 23257 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:33829
I20260811 10:07:07.084887 23331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:33829 every 8 connection(s)
I20260811 10:07:07.085503 23257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:07.087468 23332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:07.088881 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 23257
I20260811 10:07:07.089040 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:07.090070 23236 external_mini_cluster.cc:1428] Setting key 39100d107df06a6348ede76f7fd78d5a
I20260811 10:07:07.093222 23332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a: Bootstrap starting.
I20260811 10:07:07.094589 23332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:07.095676 23332 log.cc:824] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:07.096940 23332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a: No bootstrap required, opened a new log
I20260811 10:07:07.100376 23332 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f4577dea1d4352aac62a0d048d424a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33829 } }
I20260811 10:07:07.100708 23332 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:07.100845 23332 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f4577dea1d4352aac62a0d048d424a, State: Initialized, Role: FOLLOWER
I20260811 10:07:07.101069 23332 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f4577dea1d4352aac62a0d048d424a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33829 } }
I20260811 10:07:07.101253 23332 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:07.101310 23332 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:07.101364 23332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:07.102726 23332 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f4577dea1d4352aac62a0d048d424a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33829 } }
I20260811 10:07:07.103000 23332 leader_election.cc:302] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3f4577dea1d4352aac62a0d048d424a; no voters:
I20260811 10:07:07.103307 23332 leader_election.cc:288] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:07.103464 23337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:07.103834 23337 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [term 1 LEADER]: Becoming Leader. State: Replica: e3f4577dea1d4352aac62a0d048d424a, State: Running, Role: LEADER
I20260811 10:07:07.103969 23332 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:07.103981 23337 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f4577dea1d4352aac62a0d048d424a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33829 } }
I20260811 10:07:07.104673 23338 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3f4577dea1d4352aac62a0d048d424a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f4577dea1d4352aac62a0d048d424a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33829 } } }
I20260811 10:07:07.104871 23338 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:07.105358 23339 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f4577dea1d4352aac62a0d048d424a. Latest consensus state: current_term: 1 leader_uuid: "e3f4577dea1d4352aac62a0d048d424a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f4577dea1d4352aac62a0d048d424a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33829 } } }
I20260811 10:07:07.105481 23339 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:07.105700 23343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:07.107327 23343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:07.114518 23343 catalog_manager.cc:1347] Generated new cluster ID: cbd36e8e912a48379344704779838f2f
I20260811 10:07:07.114591 23343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:07.116385 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.091666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.22.177.62:33829)
0811 10:07:07.092080 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:07.092092 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:07.093653 (+ 1561us) server_negotiation.cc:407] Connection header received
0811 10:07:07.097064 (+ 3411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:07.097076 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:07.097168 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:07.097335 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:07.099678 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.101218 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.102740 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.103439 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.103600 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.110620 (+ 7020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:07.110625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:07.110626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:07.110654 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:07.113812 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.114596 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.114600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.114600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.114732 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.115049 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.115050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.115050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.115321 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:07.115548 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:07.115819 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 10:07:07.115982 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":135,"threads_started":1}
I20260811 10:07:07.142149 23343 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:07.142951 23343 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:07.147121 23343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3f4577dea1d4352aac62a0d048d424a: Generated new TSK 0
I20260811 10:07:07.147373 23343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:07.190402 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33829
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:07.286679 23363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:07.286949 23363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:07.286998 23363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:07.288305 23363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:07.288391 23363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:07.288442 23363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:07.290242 23363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:07.290650 23363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:07.290930 23363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:07.297137 23373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:07.297227 23370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:07.297137 23369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.297928 23363 server_base.cc:1034] running on GCE node
I20260811 10:07:07.298177 23363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:07.298539 23363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:07.299746 23363 hybrid_clock.cc:648] HybridClock initialized: now 1786442827299706 us; error 61 us; skew 500 ppm
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:07.303503 23363 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:07.304265 23363 webserver.cc:460] Webserver started at http://127.22.177.1:37919/ using document root <none> and password file <none>
I20260811 10:07:07.304591 23363 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:07.304690 23363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:07.304883 23363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:07.306318 23363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "29c747345c9448ea8a19d7dd3c0b2461"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "e73fbcea1188769da7d4af1d974f7595"
server_key_iv: "74850c56d76e848128d4f470c8ced786"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.306795 23363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "29c747345c9448ea8a19d7dd3c0b2461"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "e73fbcea1188769da7d4af1d974f7595"
server_key_iv: "74850c56d76e848128d4f470c8ced786"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.308682 23363 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 10:07:07.309743 23380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.310015 23363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:07.310127 23363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "29c747345c9448ea8a19d7dd3c0b2461"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "e73fbcea1188769da7d4af1d974f7595"
server_key_iv: "74850c56d76e848128d4f470c8ced786"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.310199 23363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:07.328392 23363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:07.331470 23363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:07.331722 23363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:07.332011 23363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:07.332401 23363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:07.332480 23363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.332506 23363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:07.332518 23363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.339982 23363 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:35937
I20260811 10:07:07.340111 23505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:35937 every 8 connection(s)
I20260811 10:07:07.340541 23363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:07.347978 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 23363
I20260811 10:07:07.348112 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:07.348291 23236 external_mini_cluster.cc:1428] Setting key cd1596c03ba25cb78dfe8537bd655fbf
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:07.353768 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.342107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:57499 (local address 127.22.177.62:33829)
0811 10:07:07.342254 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:07.342256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:07.342561 (+ 305us) server_negotiation.cc:407] Connection header received
0811 10:07:07.343052 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:07.343053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:07.343065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:07.343134 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:07.344451 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.344979 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.346132 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.346365 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.346435 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.349788 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:07.349790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:07.349791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:07.349801 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:07.351755 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.352443 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.352444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.352444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.352525 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.352915 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.352916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.352916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.353069 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:07.353138 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:07.353521 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 10:07:07.353692 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 10:07:07.354048 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.342006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:33829 (local address 127.22.177.1:57499)
0811 10:07:07.342258 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:07.342310 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:07.342944 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:07.343213 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:07.343216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:07.344121 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:07.344344 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:07.344350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.345102 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.345104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:07.346038 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:07.346041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.346465 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.346466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:07.346585 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.347385 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:07.347395 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:07.349563 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:07.351874 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:07.351886 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:07.351890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:07.352310 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:07.352614 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:07.352615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:07.352615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:07.352794 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:07.353154 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:07.353162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:07.353360 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 10:07:07.353484 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 10:07:07.353726 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":130,"threads_started":1}
I20260811 10:07:07.354843 23507 heartbeater.cc:347] Connected to a master server at 127.22.177.62:33829
I20260811 10:07:07.355177 23507 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:07.355692 23507 heartbeater.cc:511] Master 127.22.177.62:33829 requested a full tablet report, sending...
I20260811 10:07:07.356894 23294 ts_manager.cc:194] Registered new tserver with Master: 29c747345c9448ea8a19d7dd3c0b2461 (127.22.177.1:35937)
I20260811 10:07:07.358124 23294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:57499
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:07.381585 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33829
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:07.471030 23512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:07.471328 23512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:07.471372 23512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:07.472720 23512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:07.472810 23512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:07.472855 23512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:07.474505 23512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:07.474823 23512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:07.475087 23512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:07.480764 23522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:07.480800 23518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.480991 23512 server_base.cc:1034] running on GCE node
W20260811 10:07:07.480764 23519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.481683 23512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:07.482028 23512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:07.483260 23512 hybrid_clock.cc:648] HybridClock initialized: now 1786442827483206 us; error 68 us; skew 500 ppm
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:07.486188 23512 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:07.486658 23512 webserver.cc:460] Webserver started at http://127.22.177.2:44809/ using document root <none> and password file <none>
I20260811 10:07:07.486887 23512 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:07.486930 23512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:07.487030 23512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:07.488194 23512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "a07c673b20334175ab0ac97223c631bc"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "e41d8132a0e84f749073079dc2873dd7"
server_key_iv: "dd1525f4b29460e38aaf7e0705a88da5"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.488523 23512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "a07c673b20334175ab0ac97223c631bc"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "e41d8132a0e84f749073079dc2873dd7"
server_key_iv: "dd1525f4b29460e38aaf7e0705a88da5"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.490101 23512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:07.490979 23529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.491186 23512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:07.491281 23512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "a07c673b20334175ab0ac97223c631bc"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "e41d8132a0e84f749073079dc2873dd7"
server_key_iv: "dd1525f4b29460e38aaf7e0705a88da5"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.491344 23512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:07.507103 23512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:07.509732 23512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:07.509861 23512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:07.510071 23512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:07.510452 23512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:07.510491 23512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.510517 23512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:07.510572 23512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.517741 23512 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:34391
I20260811 10:07:07.517800 23650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:34391 every 8 connection(s)
I20260811 10:07:07.518169 23512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:07.518436 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 23512
I20260811 10:07:07.518551 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:07.518759 23236 external_mini_cluster.cc:1428] Setting key ce37ab188ac2655eba592db7e8ad17fd
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:07.530397 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.519282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:39585 (local address 127.22.177.62:33829)
0811 10:07:07.519338 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:07.519341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:07.519529 (+ 188us) server_negotiation.cc:407] Connection header received
0811 10:07:07.520087 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:07.520088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:07.520103 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:07.520173 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:07.521560 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.522149 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.523222 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.523442 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.523537 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.527021 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:07.527024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:07.527024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:07.527042 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:07.528702 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.529246 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.529247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.529248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.529326 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.529648 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.529649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.529649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.529789 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:07.529871 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:07.530190 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 10:07:07.530299 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:07:07.530601 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.519144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:33829 (local address 127.22.177.2:39585)
0811 10:07:07.519367 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:07.519420 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:07.519969 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:07.520268 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:07.520273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:07.520927 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:07.521273 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:07.521282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.522258 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.522260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:07.523111 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:07.523113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.523552 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.523553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:07.523703 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.524697 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:07.524708 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:07.526873 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:07.528868 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:07.528879 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:07.528882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:07.529148 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:07.529402 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:07.529403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:07.529403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:07.529537 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:07.529861 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:07.529869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:07.530035 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 10:07:07.530153 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 10:07:07.530346 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
I20260811 10:07:07.531235 23651 heartbeater.cc:347] Connected to a master server at 127.22.177.62:33829
I20260811 10:07:07.531356 23651 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:07.531584 23651 heartbeater.cc:511] Master 127.22.177.62:33829 requested a full tablet report, sending...
I20260811 10:07:07.532124 23294 ts_manager.cc:194] Registered new tserver with Master: a07c673b20334175ab0ac97223c631bc (127.22.177.2:34391)
I20260811 10:07:07.532662 23294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:39585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:07.554260 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33829
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:07.643380 23656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:07.643661 23656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:07.643705 23656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:07.644959 23656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:07.645044 23656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:07.645093 23656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:07.646930 23656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:07.647276 23656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:07.647550 23656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:07.653620 23661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:07.653620 23665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.653641 23656 server_base.cc:1034] running on GCE node
W20260811 10:07:07.653620 23662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:07.654152 23656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:07.654456 23656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:07.655638 23656 hybrid_clock.cc:648] HybridClock initialized: now 1786442827655628 us; error 59 us; skew 500 ppm
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:07.658607 23656 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:07.659150 23656 webserver.cc:460] Webserver started at http://127.22.177.3:42327/ using document root <none> and password file <none>
I20260811 10:07:07.659427 23656 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:07.659503 23656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:07.659660 23656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:07.660954 23656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "c9b08162a80e457ebbc3a4296e347e7e"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "ec271bb339abc10f5fc970db9b2cf7c0"
server_key_iv: "622214cf31c7924948ca9b96321d295b"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.661350 23656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "c9b08162a80e457ebbc3a4296e347e7e"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "ec271bb339abc10f5fc970db9b2cf7c0"
server_key_iv: "622214cf31c7924948ca9b96321d295b"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.663002 23656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:07.664018 23673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.664217 23656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:07.664347 23656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "c9b08162a80e457ebbc3a4296e347e7e"
format_stamp: "Formatted at 2026-08-11 10:07:07 on dist-test-slave-g350"
server_key: "ec271bb339abc10f5fc970db9b2cf7c0"
server_key_iv: "622214cf31c7924948ca9b96321d295b"
server_key_version: "encryptionkey@0"
I20260811 10:07:07.664431 23656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:07.676757 23656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:07.679193 23656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:07.679381 23656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:07.679644 23656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:07.679982 23656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:07.680040 23656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.680084 23656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:07.680112 23656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:07.686955 23656 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:44375
I20260811 10:07:07.687029 23792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:44375 every 8 connection(s)
I20260811 10:07:07.687397 23656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:07.691318 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 23656
I20260811 10:07:07.691406 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:07.691599 23236 external_mini_cluster.cc:1428] Setting key c60d31991381eb2575e35af1b106ddea
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:07.698751 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.688298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:44587 (local address 127.22.177.62:33829)
0811 10:07:07.688385 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:07.688387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:07.688525 (+ 138us) server_negotiation.cc:407] Connection header received
0811 10:07:07.689047 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:07.689048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:07.689060 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:07.689134 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:07.690222 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.690824 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.692173 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.692481 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.692568 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.695272 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:07.695274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:07.695274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:07.695282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:07.696972 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.697446 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.697447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.697448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.697509 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.697965 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.697966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.697967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.698099 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:07.698181 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:07.698567 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 10:07:07.698647 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 10:07:07.698910 23795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.688212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:33829 (local address 127.22.177.3:44587)
0811 10:07:07.688392 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:07.688436 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:07.688932 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:07.689188 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:07.689192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:07.689884 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:07.690091 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:07.690096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.690927 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.690928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:07.692032 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:07.692041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.692607 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.692608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:07.692789 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.693474 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:07.693483 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:07.695155 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:07.697080 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:07.697088 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:07.697091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:07.697354 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:07.697585 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:07.697586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:07.697586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:07.697876 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:07.698198 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:07.698206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:07.698405 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 10:07:07.698502 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 10:07:07.698661 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260811 10:07:07.699515 23793 heartbeater.cc:347] Connected to a master server at 127.22.177.62:33829
I20260811 10:07:07.699640 23793 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:07.699862 23793 heartbeater.cc:511] Master 127.22.177.62:33829 requested a full tablet report, sending...
I20260811 10:07:07.700315 23294 ts_manager.cc:194] Registered new tserver with Master: c9b08162a80e457ebbc3a4296e347e7e (127.22.177.3:44375)
I20260811 10:07:07.700682 23294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:44587
I20260811 10:07:07.704563 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:07.732659 23801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.723286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.22.177.1:35937)
0811 10:07:07.723640 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:07.723645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:07.723658 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:07:07.723687 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:07.723689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:07.723706 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:07.723848 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:07.724559 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.725326 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.726248 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.726603 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.726694 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.729379 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:07.729398 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:07.729399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:07.729406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:07.731212 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.731689 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.731693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.731693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.731780 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.732001 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.732002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.732003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.732172 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:07.732375 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:07.732423 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 10:07:07.732522 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":209,"threads_started":1}
W20260811 10:07:07.733140 23412 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54808
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:07.759867 23801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:07.751402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.22.177.1:35937)
0811 10:07:07.751496 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:07.751498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:07.751547 (+ 49us) server_negotiation.cc:407] Connection header received
0811 10:07:07.751597 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:07.751598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:07.751611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:07.751667 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:07.752378 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.752948 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.753864 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:07.754084 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:07.754145 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:07.756912 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:07.756914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:07.756914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:07.756923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:07.758555 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.758959 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.758960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.758961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.759021 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:07.759304 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:07.759305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:07.759305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:07.759435 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:07.759536 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:07.759710 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 10:07:07.759795 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 10:07:07.760708 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 23363
I20260811 10:07:07.770270 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 23512
I20260811 10:07:07.779769 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 23656
I20260811 10:07:07.789186 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 23257
2026-08-11T10:07:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2183 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23808](info): set up 2 sockets
Aug 11 10:07:07 dist-test-slave-g350 krb5kdc[23808](info): commencing operation
krb5kdc: starting...
W20260811 10:07:08.938293 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:09.093494 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:46137
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:46137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:09.190583 23822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:09.190848 23822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:09.190891 23822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:09.192130 23822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:09.192214 23822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:09.192241 23822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:09.192274 23822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:09.192301 23822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:09.193977 23822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:46137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:46137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:09.194310 23822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:09.194558 23822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:09.199810 23830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.199839 23828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.199820 23827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:09.200093 23822 server_base.cc:1034] running on GCE node
I20260811 10:07:09.200522 23822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:09.200847 23822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:09.202029 23822 hybrid_clock.cc:648] HybridClock initialized: now 1786442829202017 us; error 31 us; skew 500 ppm
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:09.204726 23822 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:09.205261 23822 webserver.cc:460] Webserver started at http://127.22.177.62:40779/ using document root <none> and password file <none>
I20260811 10:07:09.205533 23822 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:09.205617 23822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:09.205785 23822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:09.207067 23822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "fbe7e8dfba2e47e28710fa96e91a9164"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "fe4b5be14c0e52629ad7c6f318e56a17"
server_key_iv: "a8c67de3c38002734c3a9d28844b6b4b"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.207463 23822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "fbe7e8dfba2e47e28710fa96e91a9164"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "fe4b5be14c0e52629ad7c6f318e56a17"
server_key_iv: "a8c67de3c38002734c3a9d28844b6b4b"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.209002 23822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:09.210114 23838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.210356 23822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:09.210489 23822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "fbe7e8dfba2e47e28710fa96e91a9164"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "fe4b5be14c0e52629ad7c6f318e56a17"
server_key_iv: "a8c67de3c38002734c3a9d28844b6b4b"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.210573 23822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:09.224310 23822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:09.227176 23822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:09.227478 23822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:09.232100 23822 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:46137
I20260811 10:07:09.232138 23896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:46137 every 8 connection(s)
I20260811 10:07:09.232558 23822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:09.233827 23897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:09.236410 23897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164: Bootstrap starting.
I20260811 10:07:09.237030 23897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:09.237525 23897 log.cc:824] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:09.238667 23897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164: No bootstrap required, opened a new log
I20260811 10:07:09.240494 23897 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46137 } }
I20260811 10:07:09.240612 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 23822
I20260811 10:07:09.240700 23897 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:09.240746 23897 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe7e8dfba2e47e28710fa96e91a9164, State: Initialized, Role: FOLLOWER
I20260811 10:07:09.240783 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:09.240964 23236 external_mini_cluster.cc:1428] Setting key d46171cb66247848b0fdecd932cf403d
I20260811 10:07:09.240955 23897 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46137 } }
I20260811 10:07:09.241056 23897 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:09.241103 23897 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:09.241155 23897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:09.242594 23897 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46137 } }
I20260811 10:07:09.242707 23897 leader_election.cc:302] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe7e8dfba2e47e28710fa96e91a9164; no voters:
I20260811 10:07:09.242887 23897 leader_election.cc:288] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:09.242951 23902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:09.243194 23897 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:09.243292 23902 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [term 1 LEADER]: Becoming Leader. State: Replica: fbe7e8dfba2e47e28710fa96e91a9164, State: Running, Role: LEADER
I20260811 10:07:09.243548 23902 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46137 } }
I20260811 10:07:09.244091 23904 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe7e8dfba2e47e28710fa96e91a9164. Latest consensus state: current_term: 1 leader_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46137 } } }
I20260811 10:07:09.244204 23904 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:09.244654 23903 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe7e8dfba2e47e28710fa96e91a9164" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46137 } } }
I20260811 10:07:09.244745 23903 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:09.244654 23912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:09.245622 23912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:09.248409 23912 catalog_manager.cc:1347] Generated new cluster ID: 844fcb1237a34e93ad773328e65bfd98
I20260811 10:07:09.248490 23912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:09.256021 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.241570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33078 (local address 127.22.177.62:46137)
0811 10:07:09.241916 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.241929 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.241980 (+ 51us) server_negotiation.cc:407] Connection header received
0811 10:07:09.242157 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.242165 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.242230 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.242404 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.243282 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.244780 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.246242 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.247098 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.247226 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.251038 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.251044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.251044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.251061 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.253568 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.254195 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.254200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.254200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.254387 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.254643 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.254644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.254645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.255021 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.255234 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.255577 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.255765 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":171,"threads_started":1}
I20260811 10:07:09.258061 23912 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:09.258980 23912 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:09.265048 23912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe7e8dfba2e47e28710fa96e91a9164: Generated new TSK 0
I20260811 10:07:09.265295 23912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:09.306252 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46137
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:09.396076 23924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:09.396312 23924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:09.396341 23924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:09.397625 23924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:09.397680 23924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:09.397789 23924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:09.399751 23924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46137
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:09.399991 23924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:09.400206 23924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:09.405982 23932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.406008 23930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.405982 23929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:09.406510 23924 server_base.cc:1034] running on GCE node
I20260811 10:07:09.406838 23924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:09.407166 23924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:09.408382 23924 hybrid_clock.cc:648] HybridClock initialized: now 1786442829408352 us; error 50 us; skew 500 ppm
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:09.411453 23924 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:09.412007 23924 webserver.cc:460] Webserver started at http://127.22.177.1:46877/ using document root <none> and password file <none>
I20260811 10:07:09.412304 23924 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:09.412377 23924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:09.412542 23924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:09.413897 23924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "8c92b1c93db84799b0a2ab4644f65089"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "674a5d5971eb8621ed903f321a2793e2"
server_key_iv: "968a5d6fc11e89d82657cf6ce1164ba3"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.414315 23924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "8c92b1c93db84799b0a2ab4644f65089"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "674a5d5971eb8621ed903f321a2793e2"
server_key_iv: "968a5d6fc11e89d82657cf6ce1164ba3"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.415920 23924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:09.416918 23939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.417200 23924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:09.417330 23924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "8c92b1c93db84799b0a2ab4644f65089"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "674a5d5971eb8621ed903f321a2793e2"
server_key_iv: "968a5d6fc11e89d82657cf6ce1164ba3"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.417415 23924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:09.431162 23924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:09.433851 23924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:09.434038 23924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:09.434324 23924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:09.434686 23924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:09.434746 23924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.434782 23924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:09.434811 23924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.442090 23924 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:43187
I20260811 10:07:09.442139 24060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:43187 every 8 connection(s)
I20260811 10:07:09.442669 23924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:09.452883 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 23924
I20260811 10:07:09.453006 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:09.453197 23236 external_mini_cluster.cc:1428] Setting key 4d6077735bc1ac0bc7ba1518300db9c8
I20260811 10:07:09.454689 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.443742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:36485 (local address 127.22.177.62:46137)
0811 10:07:09.443839 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.443841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.444053 (+ 212us) server_negotiation.cc:407] Connection header received
0811 10:07:09.445531 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.445533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.445549 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.445691 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.446400 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.446927 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.448083 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.448301 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.448371 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.451031 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.451033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.451033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.451041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.452802 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.453462 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.453464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.453464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.453537 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.453855 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.453855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.453856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.454004 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.454078 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.454531 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.454622 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 10:07:09.454972 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.443583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:46137 (local address 127.22.177.1:36485)
0811 10:07:09.443933 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.443976 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.445398 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.445766 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.445770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.446059 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:09.446284 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.446289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.447032 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.447033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.447984 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.447987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.448442 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.448442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.448562 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.449111 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:09.449120 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:09.450915 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:09.452940 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:09.452950 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:09.452985 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:09.453373 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:09.453647 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:09.453648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:09.453648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:09.453782 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:09.454084 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:09.454093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:09.454292 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.454482 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.454686 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":120,"threads_started":1}
I20260811 10:07:09.455783 24061 heartbeater.cc:347] Connected to a master server at 127.22.177.62:46137
I20260811 10:07:09.455926 24061 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:09.456154 24061 heartbeater.cc:511] Master 127.22.177.62:46137 requested a full tablet report, sending...
I20260811 10:07:09.456765 23858 ts_manager.cc:194] Registered new tserver with Master: 8c92b1c93db84799b0a2ab4644f65089 (127.22.177.1:43187)
I20260811 10:07:09.457873 23858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:36485
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:09.486311 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46137
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:09.576050 24066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:09.576305 24066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:09.576334 24066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:09.577860 24066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:09.577936 24066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:09.577970 24066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:09.579704 24066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46137
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:09.579949 24066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:09.580325 24066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:09.586397 24066 server_base.cc:1034] running on GCE node
W20260811 10:07:09.586361 24071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.586361 24074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.586426 24072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:09.586959 24066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:09.587203 24066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:09.588379 24066 hybrid_clock.cc:648] HybridClock initialized: now 1786442829588362 us; error 33 us; skew 500 ppm
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:09.591503 24066 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:09.592064 24066 webserver.cc:460] Webserver started at http://127.22.177.2:45463/ using document root <none> and password file <none>
I20260811 10:07:09.592316 24066 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:09.592366 24066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:09.592505 24066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:09.593725 24066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "9885993cfd134d23aff2246638f948cb"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "9d44a5ae7b151c4c5f5a6eca3db156ba"
server_key_iv: "c8b621687ddc96279a54d5327c72592a"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.594091 24066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "9885993cfd134d23aff2246638f948cb"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "9d44a5ae7b151c4c5f5a6eca3db156ba"
server_key_iv: "c8b621687ddc96279a54d5327c72592a"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.595578 24066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:09.596534 24081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.596915 24066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:09.597005 24066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "9885993cfd134d23aff2246638f948cb"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "9d44a5ae7b151c4c5f5a6eca3db156ba"
server_key_iv: "c8b621687ddc96279a54d5327c72592a"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.597067 24066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:09.621439 24066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:09.624027 24066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:09.624228 24066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:09.624470 24066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:09.624785 24066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:09.624821 24066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.624845 24066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:09.624858 24066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.631297 24066 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:39957
I20260811 10:07:09.631350 24202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:39957 every 8 connection(s)
I20260811 10:07:09.631774 24066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:09.633399 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 24066
I20260811 10:07:09.633486 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:09.633699 23236 external_mini_cluster.cc:1428] Setting key b76e8f84513f3666757044e0179b7c90
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:09.644707 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.632633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:53993 (local address 127.22.177.62:46137)
0811 10:07:09.632710 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.632713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.632973 (+ 260us) server_negotiation.cc:407] Connection header received
0811 10:07:09.634242 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.634244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.634257 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.634308 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.635037 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.635554 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.636950 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.637171 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.637241 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.640276 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.640330 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.640330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.640339 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.642368 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.643072 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.643074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.643074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.643211 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.643610 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.643611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.643612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.643858 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.643977 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.644465 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.644564 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:07:09.644913 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.632602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:46137 (local address 127.22.177.2:53993)
0811 10:07:09.632831 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.632876 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.634122 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.634389 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.634394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.634715 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:09.634936 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.634940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.635662 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.635663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.636820 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.636824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.637288 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.637289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.637464 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.638158 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:09.638168 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:09.640125 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:09.642499 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:09.642513 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:09.642516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:09.642961 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:09.643298 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:09.643299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:09.643299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:09.643483 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:09.643996 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:09.644006 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:09.644290 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.644426 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.644623 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":129,"threads_started":1}
I20260811 10:07:09.645720 24203 heartbeater.cc:347] Connected to a master server at 127.22.177.62:46137
I20260811 10:07:09.645905 24203 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:09.646132 24203 heartbeater.cc:511] Master 127.22.177.62:46137 requested a full tablet report, sending...
I20260811 10:07:09.646684 23858 ts_manager.cc:194] Registered new tserver with Master: 9885993cfd134d23aff2246638f948cb (127.22.177.2:39957)
I20260811 10:07:09.647092 23858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:53993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:09.670253 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46137
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:09.762162 24208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:09.762442 24208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:09.762488 24208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:09.763759 24208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:09.763839 24208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:09.763890 24208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:09.765713 24208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46137
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:09.766012 24208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:09.766261 24208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:09.773217 24214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.773195 24217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:09.773489 24215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:09.773778 24208 server_base.cc:1034] running on GCE node
I20260811 10:07:09.774057 24208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:09.774371 24208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:09.775568 24208 hybrid_clock.cc:648] HybridClock initialized: now 1786442829775549 us; error 44 us; skew 500 ppm
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:09.778806 24208 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:09.779359 24208 webserver.cc:460] Webserver started at http://127.22.177.3:45167/ using document root <none> and password file <none>
I20260811 10:07:09.779650 24208 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:09.779706 24208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:09.779865 24208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:09.781086 24208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "fbed5d443e0e40838d8caae4656c3cfa"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "106cebe2bb0a6d09e41f51249129f161"
server_key_iv: "2c0cc99c2d4bb15ec5c1e0c709669ecc"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.781556 24208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "fbed5d443e0e40838d8caae4656c3cfa"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "106cebe2bb0a6d09e41f51249129f161"
server_key_iv: "2c0cc99c2d4bb15ec5c1e0c709669ecc"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.783502 24208 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:07:09.784502 24224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.784713 24208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:09.784878 24208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "fbed5d443e0e40838d8caae4656c3cfa"
format_stamp: "Formatted at 2026-08-11 10:07:09 on dist-test-slave-g350"
server_key: "106cebe2bb0a6d09e41f51249129f161"
server_key_iv: "2c0cc99c2d4bb15ec5c1e0c709669ecc"
server_key_version: "encryptionkey@0"
I20260811 10:07:09.784970 24208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:09.809149 24208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:09.812173 24208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:09.812386 24208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:09.812678 24208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:09.813054 24208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:09.813118 24208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.813158 24208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:09.813181 24208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:09.820721 24208 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:41347
I20260811 10:07:09.820771 24348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:41347 every 8 connection(s)
I20260811 10:07:09.821233 24208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:09.827746 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 24208
I20260811 10:07:09.827880 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:09.828095 23236 external_mini_cluster.cc:1428] Setting key 3a46c1c891204723ce357b0ebb03db4b
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:09.832559 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.822125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:59943 (local address 127.22.177.62:46137)
0811 10:07:09.822205 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.822208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.822435 (+ 227us) server_negotiation.cc:407] Connection header received
0811 10:07:09.823518 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.823520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.823533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.823605 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.824342 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.824882 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.826061 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.826268 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.826331 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.829150 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.829152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.829152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.829161 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.830907 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.831414 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.831415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.831415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.831481 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.831822 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.831823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.831824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.831955 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.832021 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.832405 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.832488 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 10:07:09.832736 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.822093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:46137 (local address 127.22.177.3:59943)
0811 10:07:09.822282 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.822330 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.823393 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.823665 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.823668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.823978 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:09.824216 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.824221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.824991 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.824992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.825970 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.825973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.826363 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.826364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.826488 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.827070 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:09.827082 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:09.829047 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:09.831013 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:09.831021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:09.831024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:09.831317 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:09.831603 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:09.831604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:09.831604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:09.831744 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:09.832040 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:09.832047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:09.832183 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.832345 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.832501 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260811 10:07:09.833436 24349 heartbeater.cc:347] Connected to a master server at 127.22.177.62:46137
I20260811 10:07:09.833590 24349 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:09.833868 24349 heartbeater.cc:511] Master 127.22.177.62:46137 requested a full tablet report, sending...
I20260811 10:07:09.834400 23858 ts_manager.cc:194] Registered new tserver with Master: fbed5d443e0e40838d8caae4656c3cfa (127.22.177.3:41347)
I20260811 10:07:09.834805 23858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:59943
I20260811 10:07:09.840583 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:07:09.850420 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.842757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.22.177.62:46137)
0811 10:07:09.842851 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.842854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.842963 (+ 109us) server_negotiation.cc:407] Connection header received
0811 10:07:09.843069 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.843071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.843083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.843162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.843920 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.844527 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.845387 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.845681 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.845795 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.847264 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.847266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.847266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.847275 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.848927 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.849356 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.849358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.849358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.849429 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.849702 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.849703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.849703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.849884 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.850055 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.850176 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.850296 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 10:07:09.853477 23858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:09.855391 23858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:09.862696 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.858375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39957 (local address 127.0.0.1:41494)
0811 10:07:09.858682 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.858730 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.858824 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.859078 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.859081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.859084 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:09.859198 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.859201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.860575 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.860576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.861496 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.861500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.862416 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.862416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.862478 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.862490 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.862551 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.862596 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260811 10:07:09.862730 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.858492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.22.177.2:39957)
0811 10:07:09.858656 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.858662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.858810 (+ 148us) server_negotiation.cc:407] Connection header received
0811 10:07:09.858917 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.858919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.858937 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.859050 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:09.859290 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.860432 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.861572 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.862319 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.862400 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.862464 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.862587 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.862636 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260811 10:07:09.862991 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.858596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43187 (local address 127.0.0.1:58636)
0811 10:07:09.858955 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.858966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.859005 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.859312 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.859314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.859316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:09.859378 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.859379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.860356 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.860357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.861727 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.861737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.862704 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.862705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.862819 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.862824 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.862882 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.862930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":15,"thread_start_us":123,"threads_started":1}
I20260811 10:07:09.863049 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.858883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.22.177.1:43187)
0811 10:07:09.859145 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.859149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.859158 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:09.859194 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.859196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.859211 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.859456 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:09.859546 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.860230 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.861834 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.862604 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.862687 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.862739 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.862920 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.862964 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":83,"threads_started":1}
I20260811 10:07:09.863849 24133 tablet_service.cc:1467] Processing CreateTablet for tablet 1f7d52cd1df24da8a05f503c4eacbe8d (DEFAULT_TABLE table=test-table [id=0e4eeb3695824dee8d7af00dd1859c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:09.863934 23991 tablet_service.cc:1467] Processing CreateTablet for tablet 1f7d52cd1df24da8a05f503c4eacbe8d (DEFAULT_TABLE table=test-table [id=0e4eeb3695824dee8d7af00dd1859c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:09.864326 23991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f7d52cd1df24da8a05f503c4eacbe8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:09.864370 24133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f7d52cd1df24da8a05f503c4eacbe8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:09.864480 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.858382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41347 (local address 127.0.0.1:57168)
0811 10:07:09.858762 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.858775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.858837 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.859158 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.859159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.859161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:09.859200 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.859201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.860544 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.860545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.861750 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.861752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.864146 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.864147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.864213 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.864217 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.864291 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.864336 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":36,"thread_start_us":97,"threads_started":1}
I20260811 10:07:09.864514 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.858489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.22.177.3:41347)
0811 10:07:09.858746 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.858750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.858925 (+ 175us) server_negotiation.cc:407] Connection header received
0811 10:07:09.858977 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.858980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.858995 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.859070 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:09.859284 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.860431 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.863233 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.863986 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.864069 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.864220 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.864319 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.864363 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":89,"threads_started":1}
I20260811 10:07:09.865293 24278 tablet_service.cc:1467] Processing CreateTablet for tablet 1f7d52cd1df24da8a05f503c4eacbe8d (DEFAULT_TABLE table=test-table [id=0e4eeb3695824dee8d7af00dd1859c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:09.865685 24278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f7d52cd1df24da8a05f503c4eacbe8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:09.867424 24367 tablet_bootstrap.cc:492] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: Bootstrap starting.
I20260811 10:07:09.868280 24367 tablet_bootstrap.cc:654] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:09.868335 24368 tablet_bootstrap.cc:492] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: Bootstrap starting.
I20260811 10:07:09.869105 24369 tablet_bootstrap.cc:492] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa: Bootstrap starting.
I20260811 10:07:09.869129 24368 tablet_bootstrap.cc:654] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:09.869045 24367 log.cc:824] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:09.869773 24368 log.cc:824] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:09.869879 24369 tablet_bootstrap.cc:654] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:09.870252 24367 tablet_bootstrap.cc:492] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: No bootstrap required, opened a new log
I20260811 10:07:09.870352 24367 ts_tablet_manager.cc:1397] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:07:09.870620 24369 log.cc:824] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:09.871165 24368 tablet_bootstrap.cc:492] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: No bootstrap required, opened a new log
I20260811 10:07:09.871244 24368 ts_tablet_manager.cc:1397] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 10:07:09.872690 24367 raft_consensus.cc:348] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.872908 24367 raft_consensus.cc:374] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:09.872948 24367 raft_consensus.cc:729] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9885993cfd134d23aff2246638f948cb, State: Initialized, Role: FOLLOWER
I20260811 10:07:09.873081 24367 consensus_queue.cc:260] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.873428 24367 ts_tablet_manager.cc:1428] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:09.873378 24368 raft_consensus.cc:348] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.873571 24203 heartbeater.cc:503] Master 127.22.177.62:46137 was elected leader, sending a full tablet report...
I20260811 10:07:09.873571 24368 raft_consensus.cc:374] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:09.873664 24368 raft_consensus.cc:729] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c92b1c93db84799b0a2ab4644f65089, State: Initialized, Role: FOLLOWER
I20260811 10:07:09.873782 24368 consensus_queue.cc:260] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.874033 24369 tablet_bootstrap.cc:492] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa: No bootstrap required, opened a new log
I20260811 10:07:09.874068 24368 ts_tablet_manager.cc:1428] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:07:09.874094 24061 heartbeater.cc:503] Master 127.22.177.62:46137 was elected leader, sending a full tablet report...
I20260811 10:07:09.874140 24369 ts_tablet_manager.cc:1397] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260811 10:07:09.875918 24369 raft_consensus.cc:348] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.876116 24369 raft_consensus.cc:374] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:09.876169 24369 raft_consensus.cc:729] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbed5d443e0e40838d8caae4656c3cfa, State: Initialized, Role: FOLLOWER
I20260811 10:07:09.876315 24369 consensus_queue.cc:260] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.876613 24369 ts_tablet_manager.cc:1428] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 10:07:09.876680 24349 heartbeater.cc:503] Master 127.22.177.62:46137 was elected leader, sending a full tablet report...
I20260811 10:07:09.882234 24374 raft_consensus.cc:482] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:09.882475 24374 raft_consensus.cc:504] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
W20260811 10:07:09.882500 24204 tablet.cc:2369] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:09.882903 24374 leader_election.cc:288] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbed5d443e0e40838d8caae4656c3cfa (127.22.177.3:41347), 9885993cfd134d23aff2246638f948cb (127.22.177.2:39957)
I20260811 10:07:09.886473 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.883250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:53055 (local address 127.22.177.2:39957)
0811 10:07:09.883324 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.883326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.883333 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:07:09.883358 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.883358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.883369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.883414 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:09.883620 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.884485 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.885682 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.886102 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.886176 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.886186 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.886380 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.886417 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:07:09.886500 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.883122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39957 (local address 127.22.177.1:53055)
0811 10:07:09.883166 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.883178 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.883233 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.883452 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.883453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.883456 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:09.883541 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.883544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.884583 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.884585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.885504 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.885508 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.886191 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.886192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.886290 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.886295 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.886365 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.886408 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 10:07:09.887085 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7d52cd1df24da8a05f503c4eacbe8d" candidate_uuid: "8c92b1c93db84799b0a2ab4644f65089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9885993cfd134d23aff2246638f948cb" is_pre_election: true
I20260811 10:07:09.887302 24153 raft_consensus.cc:2454] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c92b1c93db84799b0a2ab4644f65089 in term 0.
I20260811 10:07:09.887812 23944 leader_election.cc:302] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c92b1c93db84799b0a2ab4644f65089, 9885993cfd134d23aff2246638f948cb; no voters:
I20260811 10:07:09.888101 24374 raft_consensus.cc:2781] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:09.888216 24374 raft_consensus.cc:482] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:09.888233 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.883139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41347 (local address 127.22.177.1:43737)
0811 10:07:09.883344 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:09.883356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:09.883478 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:09.883630 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:09.883631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:09.883632 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:09.883700 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.883701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.885787 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.885788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.886915 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:09.886919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.887773 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.887774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:09.887866 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.887872 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:07:09.888014 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 10:07:09.888077 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":21,"thread_start_us":126,"threads_started":1}
I20260811 10:07:09.888244 24374 raft_consensus.cc:3037] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:09.888309 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.883242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:43737 (local address 127.22.177.3:41347)
0811 10:07:09.883288 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.883291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.883489 (+ 198us) server_negotiation.cc:407] Connection header received
0811 10:07:09.883538 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.883539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.883549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.883624 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:09.883769 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.884483 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.887012 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.887554 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.887627 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.887635 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.888046 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.888095 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:07:09.888648 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7d52cd1df24da8a05f503c4eacbe8d" candidate_uuid: "8c92b1c93db84799b0a2ab4644f65089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbed5d443e0e40838d8caae4656c3cfa" is_pre_election: true
I20260811 10:07:09.888821 24298 raft_consensus.cc:2454] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c92b1c93db84799b0a2ab4644f65089 in term 0.
I20260811 10:07:09.889330 24374 raft_consensus.cc:504] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.889493 24374 leader_election.cc:288] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [CANDIDATE]: Term 1 election: Requested vote from peers fbed5d443e0e40838d8caae4656c3cfa (127.22.177.3:41347), 9885993cfd134d23aff2246638f948cb (127.22.177.2:39957)
I20260811 10:07:09.889689 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7d52cd1df24da8a05f503c4eacbe8d" candidate_uuid: "8c92b1c93db84799b0a2ab4644f65089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbed5d443e0e40838d8caae4656c3cfa"
I20260811 10:07:09.889703 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7d52cd1df24da8a05f503c4eacbe8d" candidate_uuid: "8c92b1c93db84799b0a2ab4644f65089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9885993cfd134d23aff2246638f948cb"
I20260811 10:07:09.889784 24298 raft_consensus.cc:3037] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:09.889784 24153 raft_consensus.cc:3037] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:09.890753 24298 raft_consensus.cc:2454] T 1f7d52cd1df24da8a05f503c4eacbe8d P fbed5d443e0e40838d8caae4656c3cfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c92b1c93db84799b0a2ab4644f65089 in term 1.
I20260811 10:07:09.890762 24153 raft_consensus.cc:2454] T 1f7d52cd1df24da8a05f503c4eacbe8d P 9885993cfd134d23aff2246638f948cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c92b1c93db84799b0a2ab4644f65089 in term 1.
I20260811 10:07:09.891037 23944 leader_election.cc:302] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c92b1c93db84799b0a2ab4644f65089, 9885993cfd134d23aff2246638f948cb; no voters:
I20260811 10:07:09.891166 24374 raft_consensus.cc:2781] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:09.891399 24374 raft_consensus.cc:686] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [term 1 LEADER]: Becoming Leader. State: Replica: 8c92b1c93db84799b0a2ab4644f65089, State: Running, Role: LEADER
I20260811 10:07:09.891528 24374 consensus_queue.cc:237] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } }
I20260811 10:07:09.892485 23856 catalog_manager.cc:5495] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c92b1c93db84799b0a2ab4644f65089 (127.22.177.1). New cstate: current_term: 1 leader_uuid: "8c92b1c93db84799b0a2ab4644f65089" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbed5d443e0e40838d8caae4656c3cfa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c92b1c93db84799b0a2ab4644f65089" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9885993cfd134d23aff2246638f948cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39957 } health_report { overall_health: UNKNOWN } } }
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:09.910255 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.901534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.22.177.1:43187)
0811 10:07:09.901637 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.901639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.901738 (+ 99us) server_negotiation.cc:407] Connection header received
0811 10:07:09.901786 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.901787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.901800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.901862 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.902482 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.902980 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.903772 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.904025 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.904085 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.906911 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.906927 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.906927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.906936 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.908897 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.909326 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.909329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.909329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.909395 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.909731 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.909732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.909732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.909891 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.910003 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.910108 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.910174 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:09.938040 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.930122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58660 (local address 127.22.177.1:43187)
0811 10:07:09.930280 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.930283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.930295 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:09.930363 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.930364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.930374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.930444 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.931117 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.931618 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.932405 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.932606 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.932704 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.935167 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.935169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.935169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.935184 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.936660 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.937049 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.937050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.937051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.937120 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.937342 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.937343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.937344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.937457 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.937573 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.937840 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.937928 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
W20260811 10:07:09.943739 24062 tablet.cc:2369] T 1f7d52cd1df24da8a05f503c4eacbe8d P 8c92b1c93db84799b0a2ab4644f65089: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:07:09 dist-test-slave-g350 krb5kdc[23808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:09.968667 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:09.960374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.22.177.1:43187)
0811 10:07:09.960456 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:09.960459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:09.960594 (+ 135us) server_negotiation.cc:407] Connection header received
0811 10:07:09.960660 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:09.960661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:09.960673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:09.960736 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:09.961365 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.961965 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.962768 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:09.962969 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:09.963037 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:09.965505 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:09.965507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:09.965507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:09.965515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:09.967146 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.967576 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.967577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.967577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.967639 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:09.967943 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:09.967944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:09.967945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:09.968076 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:09.968175 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:09.968428 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 10:07:09.968506 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 10:07:09.969362 23970 tablet_service.cc:2908] Rejecting scan request for tablet 1f7d52cd1df24da8a05f503c4eacbe8d: Uninitialized: clean time has not yet been initialized
I20260811 10:07:09.970734 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 23924
I20260811 10:07:09.980408 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 24066
I20260811 10:07:09.988000 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 24208
I20260811 10:07:09.995723 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 23822
2026-08-11T10:07:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2209 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:10 dist-test-slave-g350 krb5kdc[24391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:10 dist-test-slave-g350 krb5kdc[24391](info): set up 2 sockets
Aug 11 10:07:10 dist-test-slave-g350 krb5kdc[24391](info): commencing operation
krb5kdc: starting...
W20260811 10:07:11.137084 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:11.297482 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35497
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:35497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:11.385115 24404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:11.385365 24404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:11.385392 24404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:11.386703 24404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:11.386758 24404 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 10:07:11.386772 24404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:11.386785 24404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:11.386798 24404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:11.386811 24404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:11.388444 24404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:35497
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:11.388731 24404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:11.388996 24404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:11.394774 24409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.394767 24410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.394739 24412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:11.395015 24404 server_base.cc:1034] running on GCE node
I20260811 10:07:11.395313 24404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:11.395711 24404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:11.396904 24404 hybrid_clock.cc:648] HybridClock initialized: now 1786442831396892 us; error 55 us; skew 500 ppm
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:11.399745 24404 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:11.400256 24404 webserver.cc:460] Webserver started at http://127.22.177.62:41687/ using document root <none> and password file <none>
I20260811 10:07:11.400559 24404 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:11.400638 24404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:11.400794 24404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:11.402029 24404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "c915c0124d9d47f1b9faeaded1b36e06"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "0a14848da58ea0b3dbaeebdc2a6ea9a1"
server_key_iv: "787b4203532796aae8bb58df2bb99338"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.402434 24404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "c915c0124d9d47f1b9faeaded1b36e06"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "0a14848da58ea0b3dbaeebdc2a6ea9a1"
server_key_iv: "787b4203532796aae8bb58df2bb99338"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.403970 24404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:11.404803 24419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:11.405030 24404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:11.405150 24404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "c915c0124d9d47f1b9faeaded1b36e06"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "0a14848da58ea0b3dbaeebdc2a6ea9a1"
server_key_iv: "787b4203532796aae8bb58df2bb99338"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.405385 24404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:11.417349 24404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:11.419795 24404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:11.419991 24404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:11.425246 24404 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:35497
I20260811 10:07:11.425338 24486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:35497 every 8 connection(s)
I20260811 10:07:11.425724 24404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:11.426461 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:11.428982 24487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: Bootstrap starting.
I20260811 10:07:11.429778 24487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:11.430107 24487 log.cc:824] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:11.430933 24487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: No bootstrap required, opened a new log
I20260811 10:07:11.433358 24487 raft_consensus.cc:348] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c915c0124d9d47f1b9faeaded1b36e06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35497 } }
I20260811 10:07:11.433521 24487 raft_consensus.cc:374] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:11.433544 24487 raft_consensus.cc:729] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c915c0124d9d47f1b9faeaded1b36e06, State: Initialized, Role: FOLLOWER
I20260811 10:07:11.433753 24487 consensus_queue.cc:260] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c915c0124d9d47f1b9faeaded1b36e06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35497 } }
I20260811 10:07:11.433828 24487 raft_consensus.cc:388] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:11.433858 24487 raft_consensus.cc:482] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:11.433964 24487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:11.434180 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 24404
I20260811 10:07:11.434303 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:11.434470 23236 external_mini_cluster.cc:1428] Setting key 203eaea78fa48a99f184c1f60044838b
I20260811 10:07:11.434708 24487 raft_consensus.cc:504] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c915c0124d9d47f1b9faeaded1b36e06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35497 } }
I20260811 10:07:11.434844 24487 leader_election.cc:302] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c915c0124d9d47f1b9faeaded1b36e06; no voters:
I20260811 10:07:11.435063 24487 leader_election.cc:288] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:11.435225 24492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:11.435376 24492 raft_consensus.cc:686] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [term 1 LEADER]: Becoming Leader. State: Replica: c915c0124d9d47f1b9faeaded1b36e06, State: Running, Role: LEADER
I20260811 10:07:11.435384 24487 sys_catalog.cc:564] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:11.435503 24492 consensus_queue.cc:237] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c915c0124d9d47f1b9faeaded1b36e06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35497 } }
I20260811 10:07:11.436941 24495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c915c0124d9d47f1b9faeaded1b36e06. Latest consensus state: current_term: 1 leader_uuid: "c915c0124d9d47f1b9faeaded1b36e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c915c0124d9d47f1b9faeaded1b36e06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35497 } } }
I20260811 10:07:11.437672 24495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:11.437053 24494 sys_catalog.cc:455] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c915c0124d9d47f1b9faeaded1b36e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c915c0124d9d47f1b9faeaded1b36e06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35497 } } }
I20260811 10:07:11.437855 24494 sys_catalog.cc:458] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06 [sys.catalog]: This master's current role is: LEADER
W20260811 10:07:11.438060 24511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:11.438117 24511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:11.438230 24514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:11.438406 24514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:11.442199 24514 catalog_manager.cc:1347] Generated new cluster ID: b5e8490fdad84b6e959909a958dfda66
I20260811 10:07:11.442265 24514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:11.446632 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.435019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.22.177.62:35497)
0811 10:07:11.435244 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:11.435254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:11.435359 (+ 105us) server_negotiation.cc:407] Connection header received
0811 10:07:11.435573 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:11.435582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:11.435644 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:11.435837 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:11.436717 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.438213 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.439022 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.439604 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.439740 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.442506 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:11.442510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:11.442510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:11.442523 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:11.444657 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.445124 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.445127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.445128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.445261 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.445505 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.445506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.445506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.445853 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:11.446009 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:11.446147 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 10:07:11.446295 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260811 10:07:11.468349 24514 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:11.469156 24514 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:11.477861 24514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: Generated new TSK 0
I20260811 10:07:11.478116 24514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:11.518782 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:35497
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:11.616236 24518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:11.616488 24518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:11.616518 24518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:11.617918 24518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:11.617971 24518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:11.617992 24518 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:11.618012 24518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:11.619623 24518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:35497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:11.619903 24518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:11.620167 24518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:11.625684 24518 server_base.cc:1034] running on GCE node
W20260811 10:07:11.625684 24523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.625710 24527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.625710 24525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:11.626178 24518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:11.626397 24518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:11.627578 24518 hybrid_clock.cc:648] HybridClock initialized: now 1786442831627561 us; error 32 us; skew 500 ppm
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:11.630897 24518 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:11.631536 24518 webserver.cc:460] Webserver started at http://127.22.177.1:34997/ using document root <none> and password file <none>
I20260811 10:07:11.631812 24518 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:11.631865 24518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:11.632011 24518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:11.633203 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "ba1a3aedc33445d88ce20bd8da37492a"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "88eb04dd514afb33a416d76ea39a6b8f"
server_key_iv: "f5f152ecea69be2da4db63ca47798811"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.633651 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "ba1a3aedc33445d88ce20bd8da37492a"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "88eb04dd514afb33a416d76ea39a6b8f"
server_key_iv: "f5f152ecea69be2da4db63ca47798811"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.635300 24518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:11.636269 24535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:11.636677 24518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:11.636804 24518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "ba1a3aedc33445d88ce20bd8da37492a"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "88eb04dd514afb33a416d76ea39a6b8f"
server_key_iv: "f5f152ecea69be2da4db63ca47798811"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.636896 24518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:11.648939 24518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:11.651540 24518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:11.651731 24518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:11.652339 24518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:11.652381 24518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:11.652405 24518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:11.652418 24518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:11.661144 24518 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:43275
I20260811 10:07:11.661237 24661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:43275 every 8 connection(s)
I20260811 10:07:11.661652 24518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:11.664137 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.653039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:37921 (local address 127.22.177.62:35497)
0811 10:07:11.653129 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:11.653131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:11.653322 (+ 191us) server_negotiation.cc:407] Connection header received
0811 10:07:11.653922 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:11.653924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:11.653944 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:11.654033 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:11.654692 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.655232 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.656377 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.656646 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.656738 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.660790 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:11.660791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:11.660792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:11.660802 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:11.662711 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.663125 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.663126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.663126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.663185 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.663482 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.663484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.663484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.663620 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:11.663716 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:11.663962 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 10:07:11.664052 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 10:07:11.664224 24545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.652954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.22.177.1:37921)
0811 10:07:11.653174 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:11.653220 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:11.653806 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:11.654112 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:11.654116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:11.654383 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:11.654582 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.654586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.655344 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.655345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.656270 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.656273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.656753 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.656753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.656891 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.657558 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:11.657571 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:11.660659 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:11.662811 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.662817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.662819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.663050 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.663232 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.663232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.663232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.663398 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.663702 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:11.663707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:11.663835 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 10:07:11.663920 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:07:11.664057 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260811 10:07:11.664958 24662 heartbeater.cc:347] Connected to a master server at 127.22.177.62:35497
I20260811 10:07:11.665091 24662 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:11.665315 24662 heartbeater.cc:511] Master 127.22.177.62:35497 requested a full tablet report, sending...
I20260811 10:07:11.665952 24436 ts_manager.cc:194] Registered new tserver with Master: ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275)
I20260811 10:07:11.666496 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 24518
I20260811 10:07:11.666608 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:11.666811 23236 external_mini_cluster.cc:1428] Setting key a2c12ef77b60d1198e3cfd4489b041a5
I20260811 10:07:11.667030 24436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:37921
I20260811 10:07:11.673290 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.666261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:37809 (local address 127.22.177.62:35497)
0811 10:07:11.666331 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:11.666333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:11.666408 (+ 75us) server_negotiation.cc:407] Connection header received
0811 10:07:11.666494 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:11.666495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:11.666506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:11.666572 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:11.667092 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.667966 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.669076 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.669325 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.669450 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.670675 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:11.670676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:11.670677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:11.670684 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:11.672051 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.672425 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.672426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.672427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.672493 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.672717 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.672717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.672718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.672844 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:11.672925 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:11.673086 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 10:07:11.673186 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:07:11.673303 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.666179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.22.177.1:37809)
0811 10:07:11.666345 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:11.666357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:11.666426 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:11.666577 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:11.666578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:11.666832 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:11.666909 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.666912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.668093 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.668094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.668968 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.668971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.669442 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.669443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.669542 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.670173 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:11.670178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:11.670547 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:11.672147 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.672148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.672149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.672347 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.672573 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.672574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.672575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.672642 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.672916 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:11.672916 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:11.672970 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 10:07:11.673059 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 10:07:11.673185 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:11.702742 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:35497
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:11.791592 24673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:11.791818 24673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:11.791846 24673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:11.793066 24673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:11.793115 24673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:11.793138 24673 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:11.793156 24673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:11.794668 24673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:35497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:11.794886 24673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:11.795120 24673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:11.801290 24681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.801348 24679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.801544 24678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:11.801610 24673 server_base.cc:1034] running on GCE node
I20260811 10:07:11.801945 24673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:11.802214 24673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:11.803422 24673 hybrid_clock.cc:648] HybridClock initialized: now 1786442831803422 us; error 50 us; skew 500 ppm
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:11.806519 24673 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:11.806991 24673 webserver.cc:460] Webserver started at http://127.22.177.2:40421/ using document root <none> and password file <none>
I20260811 10:07:11.807222 24673 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:11.807339 24673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:11.807466 24673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:11.808722 24673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "e35a223c4e8b408eb8b2f3dd75011f9f"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "a59db64b88f4524e8401c5eceeee6208"
server_key_iv: "0751b8225bf75d40e1df5f771f6afde1"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.809147 24673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "e35a223c4e8b408eb8b2f3dd75011f9f"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "a59db64b88f4524e8401c5eceeee6208"
server_key_iv: "0751b8225bf75d40e1df5f771f6afde1"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.810971 24673 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 10:07:11.811944 24688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:11.812155 24673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:11.812269 24673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "e35a223c4e8b408eb8b2f3dd75011f9f"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "a59db64b88f4524e8401c5eceeee6208"
server_key_iv: "0751b8225bf75d40e1df5f771f6afde1"
server_key_version: "encryptionkey@0"
I20260811 10:07:11.812433 24673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:11.831099 24673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:11.834081 24673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:11.834285 24673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:11.835029 24673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:11.835100 24673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:11.835151 24673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:11.835202 24673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:11.843854 24673 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35825
I20260811 10:07:11.843942 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35825 every 8 connection(s)
I20260811 10:07:11.844352 24673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:11.847151 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.835597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:53257 (local address 127.22.177.62:35497)
0811 10:07:11.835647 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:11.835649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:11.835903 (+ 254us) server_negotiation.cc:407] Connection header received
0811 10:07:11.836562 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:11.836563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:11.836577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:11.836658 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:11.837623 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.838475 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.840525 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.840948 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.841043 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.843737 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:11.843739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:11.843739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:11.843748 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:11.845428 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.845897 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.845899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.845899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.845966 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.846245 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.846246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.846246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.846477 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:11.846599 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:11.846946 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 10:07:11.847045 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:07:11.847343 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.835541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.22.177.2:53257)
0811 10:07:11.835730 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:11.835780 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:11.836428 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:11.836736 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:11.836740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:11.837177 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:11.837501 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.837510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.838622 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.838624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.840371 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.840376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.841104 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.841105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.841259 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.842032 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:11.842041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:11.843615 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:11.845529 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.845537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.845540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.845811 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.846049 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.846050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.846050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.846173 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.846591 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:11.846599 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:11.846785 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 10:07:11.846916 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 10:07:11.847100 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260811 10:07:11.847941 24816 heartbeater.cc:347] Connected to a master server at 127.22.177.62:35497
I20260811 10:07:11.848039 24816 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:11.848249 24816 heartbeater.cc:511] Master 127.22.177.62:35497 requested a full tablet report, sending...
I20260811 10:07:11.848737 24436 ts_manager.cc:194] Registered new tserver with Master: e35a223c4e8b408eb8b2f3dd75011f9f (127.22.177.2:35825)
I20260811 10:07:11.849260 24436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:53257
I20260811 10:07:11.849675 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 24673
I20260811 10:07:11.849778 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:11.849944 23236 external_mini_cluster.cc:1428] Setting key 8fb79c61a2de7864ae2befc6c4c44822
I20260811 10:07:11.856294 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.849165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:38975 (local address 127.22.177.62:35497)
0811 10:07:11.849217 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:11.849218 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:11.849319 (+ 101us) server_negotiation.cc:407] Connection header received
0811 10:07:11.849383 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:11.849384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:11.849395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:11.849482 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:11.849944 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.850723 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.851529 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.851840 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.851916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.853024 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:11.853026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:11.853026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:11.853034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:11.854776 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.855190 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.855191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.855191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.855260 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:11.855524 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:11.855525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:11.855526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:11.855781 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:11.855875 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:11.856080 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 10:07:11.856180 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":28}
I20260811 10:07:11.856371 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:11.849099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.22.177.2:38975)
0811 10:07:11.849240 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:11.849252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:11.849315 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:11.849492 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:11.849493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:11.849757 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:11.849851 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.849854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.850859 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.850860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.851435 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:11.851438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:11.851946 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:11.851947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:11.852015 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:11.852586 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:11.852592 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:11.852931 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:11.854882 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.854883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.854883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.855103 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.855384 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:11.855384 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:11.855385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:11.855448 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:11.855880 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:11.855881 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:11.855930 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:07:11.856045 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 10:07:11.856181 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:11.885583 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:35497
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:11.983286 24825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:11.983528 24825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:11.983556 24825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:11.984849 24825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:11.984903 24825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:11.984925 24825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:11.984943 24825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:11.986644 24825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:35497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:11.986958 24825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:11.987237 24825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:11.993528 24833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.993548 24831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:11.993528 24830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:11.993548 24825 server_base.cc:1034] running on GCE node
I20260811 10:07:11.993991 24825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:11.994294 24825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:11.995477 24825 hybrid_clock.cc:648] HybridClock initialized: now 1786442831995463 us; error 50 us; skew 500 ppm
Aug 11 10:07:11 dist-test-slave-g350 krb5kdc[24391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:11.998829 24825 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:11.999346 24825 webserver.cc:460] Webserver started at http://127.22.177.3:41421/ using document root <none> and password file <none>
I20260811 10:07:11.999624 24825 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:11.999696 24825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:11.999861 24825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:12.000982 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "ec0a3a6096014669b0606e89f6513027"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "c2de39e41d81dfa84c9ede04b01ebde0"
server_key_iv: "68433c6de87aab70f2b4fa7fedf1a923"
server_key_version: "encryptionkey@0"
I20260811 10:07:12.001410 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "ec0a3a6096014669b0606e89f6513027"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "c2de39e41d81dfa84c9ede04b01ebde0"
server_key_iv: "68433c6de87aab70f2b4fa7fedf1a923"
server_key_version: "encryptionkey@0"
I20260811 10:07:12.003000 24825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:12.003930 24845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:12.004232 24825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:12.004379 24825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "ec0a3a6096014669b0606e89f6513027"
format_stamp: "Formatted at 2026-08-11 10:07:11 on dist-test-slave-g350"
server_key: "c2de39e41d81dfa84c9ede04b01ebde0"
server_key_iv: "68433c6de87aab70f2b4fa7fedf1a923"
server_key_version: "encryptionkey@0"
I20260811 10:07:12.004479 24825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:12.018306 24825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:12.020879 24825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:12.021070 24825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:12.021843 24825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:12.021908 24825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:12.021950 24825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:12.021978 24825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:12.029776 24825 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:41637
I20260811 10:07:12.029838 24970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:41637 every 8 connection(s)
I20260811 10:07:12.030253 24825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:12.034420 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.022396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:59095 (local address 127.22.177.62:35497)
0811 10:07:12.022476 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.022479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.022730 (+ 251us) server_negotiation.cc:407] Connection header received
0811 10:07:12.023417 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.023419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.023434 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.023497 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:12.024612 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.025457 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.027452 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.027761 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.027868 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.031087 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:12.031089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:12.031089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:12.031106 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:12.032771 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.033255 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.033256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.033257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.033326 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.033645 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.033647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.033647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.033793 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:12.033886 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.034199 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.034320 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:07:12.034621 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.022376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.22.177.3:59095)
0811 10:07:12.022571 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.022617 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.023197 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.023604 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.023610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.024095 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:12.024494 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.024501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.025586 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.025587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.027314 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.027320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.027882 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.027883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.028090 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.028910 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:12.028935 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:12.030945 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:12.032887 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:12.032895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:12.032897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:12.033158 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:12.033407 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:12.033408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:12.033409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:12.033537 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:12.033874 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:12.033882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:12.034041 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.034160 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.034351 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260811 10:07:12.034855 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 24825
I20260811 10:07:12.034956 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:12.035136 23236 external_mini_cluster.cc:1428] Setting key e8f413ce37abf58266b4f42e9a3497ca
I20260811 10:07:12.035394 24971 heartbeater.cc:347] Connected to a master server at 127.22.177.62:35497
I20260811 10:07:12.035508 24971 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:12.035727 24971 heartbeater.cc:511] Master 127.22.177.62:35497 requested a full tablet report, sending...
I20260811 10:07:12.036180 24437 ts_manager.cc:194] Registered new tserver with Master: ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637)
I20260811 10:07:12.036751 24437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:59095
I20260811 10:07:12.043256 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.036989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.22.177.3:34973)
0811 10:07:12.037136 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.037147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.037236 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.037356 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.037356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.037555 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:12.037668 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.037670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.038345 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.038346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.038874 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.038876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.039294 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.039295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.039378 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.039923 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:12.039928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:12.040267 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:12.042097 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:12.042099 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:12.042099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:12.042295 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:12.042513 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:12.042514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:12.042515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:12.042584 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:12.042870 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:12.042872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:12.042939 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.043016 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.043143 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260811 10:07:12.043246 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.037032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:34973 (local address 127.22.177.62:35497)
0811 10:07:12.037078 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.037080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.037220 (+ 140us) server_negotiation.cc:407] Connection header received
0811 10:07:12.037291 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.037292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.037302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.037352 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:12.037753 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.038236 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.038964 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.039212 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.039295 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.040367 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:12.040368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:12.040369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:12.040376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:12.041988 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.042373 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.042374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.042375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.042435 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.042656 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.042657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.042658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.042792 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:12.042869 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.043055 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.043140 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:07:12.047361 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[24391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:12.078953 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.066613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50448 (local address 127.22.177.62:35497)
0811 10:07:12.066696 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.066698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.066705 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:07:12.066731 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.066732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.066742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.066831 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:12.067455 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.067964 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.068730 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.068928 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.069033 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.075963 (+ 6930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:12.075965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:12.075965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:12.075975 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:12.077570 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.078036 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.078038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.078039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.078106 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.078353 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.078354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.078355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.078486 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:12.078607 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.078745 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.078841 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 10:07:12.081075 24437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:12.083060 24437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:12.090546 24991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.086143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41637 (local address 127.0.0.1:45128)
0811 10:07:12.086287 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.086343 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.086429 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.086701 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.086705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.086707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:12.086794 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.086797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.088073 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.088075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.089102 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.089105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.090140 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.090141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.090240 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.090256 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.090374 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.090438 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260811 10:07:12.090544 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.086279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.22.177.3:41637)
0811 10:07:12.086489 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.086494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.086526 (+ 32us) server_negotiation.cc:407] Connection header received
0811 10:07:12.086573 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.086575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.086591 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.086676 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:12.087028 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.087975 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.089213 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.090040 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.090139 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.090208 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.090397 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.090437 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":134,"threads_started":1}
I20260811 10:07:12.091017 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.086637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35825 (local address 127.0.0.1:57590)
0811 10:07:12.086939 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.086948 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.086989 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.087308 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.087309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.087312 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:12.087363 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.087364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.088787 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.088789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.089808 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.089811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.090768 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.090769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.090833 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.090838 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.090905 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.090962 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260811 10:07:12.091457 24901 tablet_service.cc:1467] Processing CreateTablet for tablet 34861104ac514a5596e6719666bd74aa (DEFAULT_TABLE table=test-table [id=9a13535302a241febeb5ca4132667eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:12.091702 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.086431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43275 (local address 127.0.0.1:51240)
0811 10:07:12.086907 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.086918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.086963 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.087489 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.087490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.087491 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:12.087521 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.087522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.088761 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.088762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.090006 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.090010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.091501 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.091501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.091550 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.091554 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.091599 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.091629 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":37,"thread_start_us":86,"threads_started":1}
I20260811 10:07:12.091765 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.087031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.22.177.1:43275)
0811 10:07:12.087229 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.087233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.087245 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:12.087283 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.087285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.087300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.087499 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:12.087573 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.088654 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.090113 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.091404 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.091501 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.091560 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.091613 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.091659 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260811 10:07:12.091943 24901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34861104ac514a5596e6719666bd74aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.092394 24588 tablet_service.cc:1467] Processing CreateTablet for tablet 34861104ac514a5596e6719666bd74aa (DEFAULT_TABLE table=test-table [id=9a13535302a241febeb5ca4132667eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:12.092751 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34861104ac514a5596e6719666bd74aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.095764 24997 tablet_bootstrap.cc:492] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Bootstrap starting.
I20260811 10:07:12.096194 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.086762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.22.177.2:35825)
0811 10:07:12.087123 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.087127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.087158 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:07:12.087200 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.087202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.087217 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.087303 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:12.087456 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.088663 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.089924 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.090665 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.090766 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.090820 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.095985 (+ 5165us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.096061 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":126,"threads_started":1}
I20260811 10:07:12.096642 25000 tablet_bootstrap.cc:492] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Bootstrap starting.
I20260811 10:07:12.096886 24747 tablet_service.cc:1467] Processing CreateTablet for tablet 34861104ac514a5596e6719666bd74aa (DEFAULT_TABLE table=test-table [id=9a13535302a241febeb5ca4132667eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:12.097229 24747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34861104ac514a5596e6719666bd74aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.097385 24997 tablet_bootstrap.cc:654] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.097385 25000 tablet_bootstrap.cc:654] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.097841 24997 log.cc:824] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:12.097841 25000 log.cc:824] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:12.098634 25000 tablet_bootstrap.cc:492] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: No bootstrap required, opened a new log
I20260811 10:07:12.098693 25000 ts_tablet_manager.cc:1397] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:07:12.099193 24997 tablet_bootstrap.cc:492] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: No bootstrap required, opened a new log
I20260811 10:07:12.099299 24997 ts_tablet_manager.cc:1397] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 10:07:12.100641 25003 tablet_bootstrap.cc:492] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Bootstrap starting.
I20260811 10:07:12.100908 25000 raft_consensus.cc:348] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.101097 25000 raft_consensus.cc:374] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.101138 25000 raft_consensus.cc:729] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba1a3aedc33445d88ce20bd8da37492a, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.101267 25000 consensus_queue.cc:260] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.101426 25003 tablet_bootstrap.cc:654] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.101578 25000 ts_tablet_manager.cc:1428] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:12.101516 24997 raft_consensus.cc:348] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.101729 24997 raft_consensus.cc:374] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.101779 24997 raft_consensus.cc:729] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec0a3a6096014669b0606e89f6513027, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.101816 24662 heartbeater.cc:503] Master 127.22.177.62:35497 was elected leader, sending a full tablet report...
I20260811 10:07:12.101862 25003 log.cc:824] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:12.101908 24997 consensus_queue.cc:260] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.102213 24997 ts_tablet_manager.cc:1428] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:12.102268 24971 heartbeater.cc:503] Master 127.22.177.62:35497 was elected leader, sending a full tablet report...
I20260811 10:07:12.102814 25003 tablet_bootstrap.cc:492] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: No bootstrap required, opened a new log
I20260811 10:07:12.102892 25003 ts_tablet_manager.cc:1397] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:07:12.104526 25003 raft_consensus.cc:348] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.104664 25003 raft_consensus.cc:374] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.104686 25003 raft_consensus.cc:729] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35a223c4e8b408eb8b2f3dd75011f9f, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.104825 25003 consensus_queue.cc:260] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.105108 25003 ts_tablet_manager.cc:1428] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 10:07:12.105178 24816 heartbeater.cc:503] Master 127.22.177.62:35497 was elected leader, sending a full tablet report...
W20260811 10:07:12.162537 24664 tablet.cc:2369] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:07:12.281078 24972 tablet.cc:2369] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:07:12.345453 24817 tablet.cc:2369] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:12.457152 25007 raft_consensus.cc:482] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:12.457361 25007 raft_consensus.cc:504] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.457939 25007 leader_election.cc:288] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637), ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275)
I20260811 10:07:12.461100 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.458484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:42037 (local address 127.22.177.3:41637)
0811 10:07:12.458557 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.458559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.458570 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:07:12.458601 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.458602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.458612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.458671 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:12.458864 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.459343 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.460350 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.460713 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.460809 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.460818 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.460975 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.461014 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 10:07:12.461135 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.458342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43275 (local address 127.22.177.2:42999)
0811 10:07:12.458504 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.458516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.458596 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.458800 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.458801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.458803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:12.458873 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.458875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.459523 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.459524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.460276 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.460278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.460842 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.460844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.460923 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.460927 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.460997 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.461041 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":46,"threads_started":1}
I20260811 10:07:12.461118 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.458590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:42999 (local address 127.22.177.1:43275)
0811 10:07:12.458662 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.458664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.458673 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:12.458708 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.458709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.458717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.458792 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:12.458949 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.459424 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.460377 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.460750 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.460842 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.460851 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.461003 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.461042 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:07:12.461145 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.458129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41637 (local address 127.22.177.2:42037)
0811 10:07:12.458282 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.458296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.458356 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.458701 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.458703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.458706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:12.458779 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.458782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.459435 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.459436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.460232 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.460244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.460810 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.460811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.460884 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.460887 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.460963 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.461014 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260811 10:07:12.461584 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34861104ac514a5596e6719666bd74aa" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1a3aedc33445d88ce20bd8da37492a" is_pre_election: true
I20260811 10:07:12.461813 24612 raft_consensus.cc:2454] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 0.
I20260811 10:07:12.461779 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34861104ac514a5596e6719666bd74aa" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0a3a6096014669b0606e89f6513027" is_pre_election: true
I20260811 10:07:12.461911 24922 raft_consensus.cc:2454] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 0.
I20260811 10:07:12.462069 24690 leader_election.cc:302] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba1a3aedc33445d88ce20bd8da37492a, e35a223c4e8b408eb8b2f3dd75011f9f; no voters:
I20260811 10:07:12.462216 25007 raft_consensus.cc:2781] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:12.462312 25007 raft_consensus.cc:482] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:12.462350 25007 raft_consensus.cc:3037] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.463133 25007 raft_consensus.cc:504] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.463300 25007 leader_election.cc:288] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 election: Requested vote from peers ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637), ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275)
I20260811 10:07:12.463538 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34861104ac514a5596e6719666bd74aa" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0a3a6096014669b0606e89f6513027"
I20260811 10:07:12.463554 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34861104ac514a5596e6719666bd74aa" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1a3aedc33445d88ce20bd8da37492a"
I20260811 10:07:12.463640 24922 raft_consensus.cc:3037] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.463645 24612 raft_consensus.cc:3037] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.464403 24612 raft_consensus.cc:2454] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 1.
I20260811 10:07:12.464469 24922 raft_consensus.cc:2454] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 1.
I20260811 10:07:12.464675 24690 leader_election.cc:302] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e35a223c4e8b408eb8b2f3dd75011f9f, ec0a3a6096014669b0606e89f6513027; no voters:
I20260811 10:07:12.464844 25007 raft_consensus.cc:2781] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:12.464998 25007 raft_consensus.cc:686] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 LEADER]: Becoming Leader. State: Replica: e35a223c4e8b408eb8b2f3dd75011f9f, State: Running, Role: LEADER
I20260811 10:07:12.465113 25007 consensus_queue.cc:237] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } }
I20260811 10:07:12.466074 24436 catalog_manager.cc:5495] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to e35a223c4e8b408eb8b2f3dd75011f9f (127.22.177.2). New cstate: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } health_report { overall_health: HEALTHY } } }
I20260811 10:07:12.539592 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.536794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57602 (local address 127.22.177.2:35825)
0811 10:07:12.536896 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.536902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.536914 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:12.536958 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.536958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.536978 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.537040 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:12.537186 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.537758 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.538681 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.538879 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.538950 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.539188 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:12.539299 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:12.539364 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.539445 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.539467 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 10:07:12.541831 24612 raft_consensus.cc:1261] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Refusing update from remote peer e35a223c4e8b408eb8b2f3dd75011f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:12.541855 24922 raft_consensus.cc:1261] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Refusing update from remote peer e35a223c4e8b408eb8b2f3dd75011f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:12.542261 25010 consensus_queue.cc:1035] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:12.542464 25007 consensus_queue.cc:1035] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:12.545367 25014 mvcc.cc:205] Tried to move back new op lower bound from 7317269842087907328 to 7317269841777758208. Current Snapshot: MvccSnapshot[applied={T|T < 7317269842087907328}]
I20260811 10:07:12.545625 25015 mvcc.cc:205] Tried to move back new op lower bound from 7317269842087907328 to 7317269841777758208. Current Snapshot: MvccSnapshot[applied={T|T < 7317269842087907328}]
I20260811 10:07:12.549685 24437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:12.549809 24437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:12.550827 24437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: Sending DeleteTablet for 3 replicas of tablet 34861104ac514a5596e6719666bd74aa
I20260811 10:07:12.551216 24901 tablet_service.cc:1514] Processing DeleteTablet for tablet 34861104ac514a5596e6719666bd74aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:12 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:45128
I20260811 10:07:12.551350 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet 34861104ac514a5596e6719666bd74aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:12 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:51240
I20260811 10:07:12.551399 24747 tablet_service.cc:1514] Processing DeleteTablet for tablet 34861104ac514a5596e6719666bd74aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:12 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:57590
I20260811 10:07:12.551546 25022 tablet_replica.cc:330] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: stopping tablet replica
I20260811 10:07:12.551692 25022 raft_consensus.cc:2229] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:12.551856 25022 raft_consensus.cc:2258] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:12.551870 25024 tablet_replica.cc:330] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: stopping tablet replica
I20260811 10:07:12.551887 24437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 10:07:12.552002 25024 raft_consensus.cc:2229] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 10:07:12.552047 24437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:12.552103 25024 raft_consensus.cc:2258] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:12.552279 25026 tablet_replica.cc:330] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: stopping tablet replica
I20260811 10:07:12.552562 25022 ts_tablet_manager.cc:1905] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:12.552403 25026 raft_consensus.cc:2229] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:12.552817 25026 raft_consensus.cc:2258] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:12.553504 24901 tablet_service.cc:1467] Processing CreateTablet for tablet 463db5beb904484bb21cd87d5d659dca (DEFAULT_TABLE table=test-table [id=b391bc9cb3394106b60c95df35b70ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:12.553508 24588 tablet_service.cc:1467] Processing CreateTablet for tablet 463db5beb904484bb21cd87d5d659dca (DEFAULT_TABLE table=test-table [id=b391bc9cb3394106b60c95df35b70ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:12.553670 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463db5beb904484bb21cd87d5d659dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.553822 24747 tablet_service.cc:1467] Processing CreateTablet for tablet 463db5beb904484bb21cd87d5d659dca (DEFAULT_TABLE table=test-table [id=b391bc9cb3394106b60c95df35b70ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:12.553932 24747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463db5beb904484bb21cd87d5d659dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.555179 25000 tablet_bootstrap.cc:492] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Bootstrap starting.
I20260811 10:07:12.555298 25003 tablet_bootstrap.cc:492] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Bootstrap starting.
I20260811 10:07:12.555673 25000 tablet_bootstrap.cc:654] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.555790 25003 tablet_bootstrap.cc:654] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.556156 24901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463db5beb904484bb21cd87d5d659dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.556428 25000 tablet_bootstrap.cc:492] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: No bootstrap required, opened a new log
I20260811 10:07:12.556481 25000 ts_tablet_manager.cc:1397] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:12.556663 25000 raft_consensus.cc:348] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.556746 25000 raft_consensus.cc:374] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.556790 25000 raft_consensus.cc:729] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba1a3aedc33445d88ce20bd8da37492a, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.556852 25000 consensus_queue.cc:260] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.556998 25000 ts_tablet_manager.cc:1428] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:12.557937 24997 tablet_bootstrap.cc:492] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Bootstrap starting.
I20260811 10:07:12.558456 25003 tablet_bootstrap.cc:492] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: No bootstrap required, opened a new log
I20260811 10:07:12.558530 24997 tablet_bootstrap.cc:654] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.558530 25003 ts_tablet_manager.cc:1397] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 10:07:12.558735 25003 raft_consensus.cc:348] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.557937 25022 ts_tablet_manager.cc:1918] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:12.558872 25003 raft_consensus.cc:374] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.558905 25003 raft_consensus.cc:729] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35a223c4e8b408eb8b2f3dd75011f9f, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.558939 25022 log.cc:1196] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/34861104ac514a5596e6719666bd74aa
I20260811 10:07:12.558959 25003 consensus_queue.cc:260] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.559073 25003 ts_tablet_manager.cc:1428] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:12.559043 25024 ts_tablet_manager.cc:1905] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:12.559275 25022 ts_tablet_manager.cc:1939] T 34861104ac514a5596e6719666bd74aa P ec0a3a6096014669b0606e89f6513027: Deleting consensus metadata
I20260811 10:07:12.559768 24421 catalog_manager.cc:4841] TS ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637): tablet 34861104ac514a5596e6719666bd74aa (table test-table [id=9a13535302a241febeb5ca4132667eec]) successfully deleted
I20260811 10:07:12.561632 25026 ts_tablet_manager.cc:1905] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:12.561632 25024 ts_tablet_manager.cc:1918] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:12.562105 24997 tablet_bootstrap.cc:492] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: No bootstrap required, opened a new log
I20260811 10:07:12.562171 24997 ts_tablet_manager.cc:1397] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 10:07:12.562177 25024 log.cc:1196] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/34861104ac514a5596e6719666bd74aa
I20260811 10:07:12.562510 25024 ts_tablet_manager.cc:1939] T 34861104ac514a5596e6719666bd74aa P ba1a3aedc33445d88ce20bd8da37492a: Deleting consensus metadata
I20260811 10:07:12.562702 24997 raft_consensus.cc:348] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.562826 24997 raft_consensus.cc:374] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.562860 24997 raft_consensus.cc:729] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec0a3a6096014669b0606e89f6513027, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.562964 24997 consensus_queue.cc:260] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.563082 24997 ts_tablet_manager.cc:1428] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:12.563840 24421 catalog_manager.cc:4841] TS ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275): tablet 34861104ac514a5596e6719666bd74aa (table test-table [id=9a13535302a241febeb5ca4132667eec]) successfully deleted
I20260811 10:07:12.564281 25026 ts_tablet_manager.cc:1918] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:12.564357 25026 log.cc:1196] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/34861104ac514a5596e6719666bd74aa
I20260811 10:07:12.564647 25026 ts_tablet_manager.cc:1939] T 34861104ac514a5596e6719666bd74aa P e35a223c4e8b408eb8b2f3dd75011f9f: Deleting consensus metadata
I20260811 10:07:12.565284 24421 catalog_manager.cc:4841] TS e35a223c4e8b408eb8b2f3dd75011f9f (127.22.177.2:35825): tablet 34861104ac514a5596e6719666bd74aa (table test-table [id=9a13535302a241febeb5ca4132667eec]) successfully deleted
I20260811 10:07:12.639710 25010 raft_consensus.cc:482] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:12.639853 25010 raft_consensus.cc:504] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.640077 25010 leader_election.cc:288] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275), ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637)
I20260811 10:07:12.640420 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463db5beb904484bb21cd87d5d659dca" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0a3a6096014669b0606e89f6513027" is_pre_election: true
I20260811 10:07:12.640439 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463db5beb904484bb21cd87d5d659dca" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1a3aedc33445d88ce20bd8da37492a" is_pre_election: true
I20260811 10:07:12.640537 24922 raft_consensus.cc:2454] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 0.
I20260811 10:07:12.640563 24612 raft_consensus.cc:2454] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 0.
I20260811 10:07:12.640729 24690 leader_election.cc:302] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba1a3aedc33445d88ce20bd8da37492a, e35a223c4e8b408eb8b2f3dd75011f9f; no voters:
I20260811 10:07:12.640884 25010 raft_consensus.cc:2781] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:12.640951 25010 raft_consensus.cc:482] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:12.640975 25010 raft_consensus.cc:3037] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.641989 25010 raft_consensus.cc:504] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.642132 25010 leader_election.cc:288] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 election: Requested vote from peers ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275), ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637)
I20260811 10:07:12.642333 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463db5beb904484bb21cd87d5d659dca" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1a3aedc33445d88ce20bd8da37492a"
I20260811 10:07:12.642424 24612 raft_consensus.cc:3037] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.642422 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463db5beb904484bb21cd87d5d659dca" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0a3a6096014669b0606e89f6513027"
I20260811 10:07:12.642520 24922 raft_consensus.cc:3037] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.643167 24922 raft_consensus.cc:2454] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 1.
I20260811 10:07:12.643280 24612 raft_consensus.cc:2454] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 1.
I20260811 10:07:12.643383 24690 leader_election.cc:302] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e35a223c4e8b408eb8b2f3dd75011f9f, ec0a3a6096014669b0606e89f6513027; no voters:
I20260811 10:07:12.643479 25010 raft_consensus.cc:2781] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:12.643536 25010 raft_consensus.cc:686] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 LEADER]: Becoming Leader. State: Replica: e35a223c4e8b408eb8b2f3dd75011f9f, State: Running, Role: LEADER
I20260811 10:07:12.643591 25010 consensus_queue.cc:237] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.644289 24437 catalog_manager.cc:5495] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to e35a223c4e8b408eb8b2f3dd75011f9f (127.22.177.2). New cstate: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } health_report { overall_health: UNKNOWN } } }
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:12.686643 25038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.678599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.22.177.62:35497)
0811 10:07:12.678793 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.678796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.678819 (+ 23us) server_negotiation.cc:407] Connection header received
0811 10:07:12.678849 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.678849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.678859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.678943 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:12.679511 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.680077 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.680874 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.681076 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.681144 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.683506 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:12.683508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:12.683508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:12.683517 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:12.685103 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.685709 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.685710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.685711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.685770 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.686034 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.686035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.686035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.686178 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:12.686246 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.686479 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.686570 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":168,"threads_started":1}
I20260811 10:07:12.686733 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.678536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35497 (local address 127.0.0.1:50464)
0811 10:07:12.678720 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.678734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.678802 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.678989 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.678990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.679311 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:12.679390 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.679393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.680167 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.680168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.680761 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.680764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.681160 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.681161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.681209 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.681841 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:12.681847 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:12.683331 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:12.685210 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:12.685289 (+ 79us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:12.685291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:12.685541 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:12.685856 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:12.685857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:12.685858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:12.685931 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:12.686321 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:12.686329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:12.686381 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.686448 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.686657 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":81,"thread_start_us":112,"threads_started":1}
I20260811 10:07:12.689493 24437 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:50464:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 10:07:12.689813 24437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:12.691672 24747 tablet_service.cc:1467] Processing CreateTablet for tablet b3a681cfedff49d5bb41a2602f9fb9a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3c0efcb4a3245c1adc2e38742cabc9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:12.691655 24588 tablet_service.cc:1467] Processing CreateTablet for tablet b3a681cfedff49d5bb41a2602f9fb9a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3c0efcb4a3245c1adc2e38742cabc9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:12.691718 24901 tablet_service.cc:1467] Processing CreateTablet for tablet b3a681cfedff49d5bb41a2602f9fb9a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3c0efcb4a3245c1adc2e38742cabc9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:12.691857 24901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3a681cfedff49d5bb41a2602f9fb9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.691869 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3a681cfedff49d5bb41a2602f9fb9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.691885 24747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3a681cfedff49d5bb41a2602f9fb9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:12.693171 25003 tablet_bootstrap.cc:492] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: Bootstrap starting.
I20260811 10:07:12.693465 24997 tablet_bootstrap.cc:492] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: Bootstrap starting.
I20260811 10:07:12.693570 25003 tablet_bootstrap.cc:654] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.693878 24997 tablet_bootstrap.cc:654] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.693953 25000 tablet_bootstrap.cc:492] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: Bootstrap starting.
I20260811 10:07:12.694327 25003 tablet_bootstrap.cc:492] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: No bootstrap required, opened a new log
I20260811 10:07:12.694376 25003 ts_tablet_manager.cc:1397] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:12.694378 25000 tablet_bootstrap.cc:654] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:12.694501 25003 raft_consensus.cc:348] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.694582 25003 raft_consensus.cc:374] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.694582 24997 tablet_bootstrap.cc:492] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: No bootstrap required, opened a new log
I20260811 10:07:12.694605 25003 raft_consensus.cc:729] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35a223c4e8b408eb8b2f3dd75011f9f, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.694625 24997 ts_tablet_manager.cc:1397] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:12.694662 25003 consensus_queue.cc:260] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.694775 25003 ts_tablet_manager.cc:1428] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:12.694772 24997 raft_consensus.cc:348] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.694844 24997 raft_consensus.cc:374] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.694864 24997 raft_consensus.cc:729] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec0a3a6096014669b0606e89f6513027, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.694840 25007 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.694944 25007 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.694938 24997 consensus_queue.cc:260] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.695058 24997 ts_tablet_manager.cc:1428] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:12.695151 25000 tablet_bootstrap.cc:492] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: No bootstrap required, opened a new log
I20260811 10:07:12.695197 25000 ts_tablet_manager.cc:1397] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:12.695135 25005 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.695255 25005 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.695351 25000 raft_consensus.cc:348] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.695429 25000 raft_consensus.cc:374] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:12.695456 25000 raft_consensus.cc:729] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba1a3aedc33445d88ce20bd8da37492a, State: Initialized, Role: FOLLOWER
I20260811 10:07:12.695508 25000 consensus_queue.cc:260] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.695607 25000 ts_tablet_manager.cc:1428] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:12.695672 25004 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.695767 25004 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.767213 25007 raft_consensus.cc:482] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:12.767364 25007 raft_consensus.cc:504] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.767534 25007 leader_election.cc:288] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275), ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637)
I20260811 10:07:12.767844 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a681cfedff49d5bb41a2602f9fb9a3" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1a3aedc33445d88ce20bd8da37492a" is_pre_election: true
I20260811 10:07:12.767938 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a681cfedff49d5bb41a2602f9fb9a3" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0a3a6096014669b0606e89f6513027" is_pre_election: true
I20260811 10:07:12.768002 24612 raft_consensus.cc:2454] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 0.
I20260811 10:07:12.768052 24922 raft_consensus.cc:2454] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 0.
I20260811 10:07:12.768190 24690 leader_election.cc:302] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba1a3aedc33445d88ce20bd8da37492a, e35a223c4e8b408eb8b2f3dd75011f9f; no voters:
I20260811 10:07:12.768323 25007 raft_consensus.cc:2781] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:12.768404 25007 raft_consensus.cc:482] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:12.768435 25007 raft_consensus.cc:3037] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.769202 25007 raft_consensus.cc:504] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.769377 25007 leader_election.cc:288] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 election: Requested vote from peers ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275), ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637)
I20260811 10:07:12.769644 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a681cfedff49d5bb41a2602f9fb9a3" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1a3aedc33445d88ce20bd8da37492a"
I20260811 10:07:12.769742 24612 raft_consensus.cc:3037] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.769667 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a681cfedff49d5bb41a2602f9fb9a3" candidate_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0a3a6096014669b0606e89f6513027"
I20260811 10:07:12.769795 24922 raft_consensus.cc:3037] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:12.770381 24612 raft_consensus.cc:2454] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 1.
I20260811 10:07:12.770514 24922 raft_consensus.cc:2454] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35a223c4e8b408eb8b2f3dd75011f9f in term 1.
I20260811 10:07:12.770572 24690 leader_election.cc:302] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba1a3aedc33445d88ce20bd8da37492a, e35a223c4e8b408eb8b2f3dd75011f9f; no voters:
I20260811 10:07:12.770699 25007 raft_consensus.cc:2781] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:12.770752 25007 raft_consensus.cc:686] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 LEADER]: Becoming Leader. State: Replica: e35a223c4e8b408eb8b2f3dd75011f9f, State: Running, Role: LEADER
I20260811 10:07:12.770793 25007 consensus_queue.cc:237] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } }
I20260811 10:07:12.771195 25010 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: TxnStatusTablet state changed. Reason: New leader e35a223c4e8b408eb8b2f3dd75011f9f. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.771292 25010 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:12.771557 25043 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 10:07:12.771587 24437 catalog_manager.cc:5495] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to e35a223c4e8b408eb8b2f3dd75011f9f (127.22.177.2). New cstate: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:12.771711 25043 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 10:07:12.793013 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.790311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35825 (local address 127.0.0.1:57606)
0811 10:07:12.790401 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.790439 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.790501 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.790707 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.790708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.790709 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:12.790778 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.790780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.791455 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.791456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.792084 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.792086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.792507 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.792509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.792577 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.792597 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:12.792813 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:12.792825 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.792880 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.792940 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 10:07:12.793005 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.790407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.22.177.2:35825)
0811 10:07:12.790491 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.790493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.790505 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:12.790581 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.790582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.790597 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.790693 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:12.790859 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.791354 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.792196 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.792404 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.792502 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.792693 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:12.792741 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:12.792811 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.792882 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.792912 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 10:07:12.794404 24612 raft_consensus.cc:1261] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Refusing update from remote peer e35a223c4e8b408eb8b2f3dd75011f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:12.794404 24922 raft_consensus.cc:1261] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Refusing update from remote peer e35a223c4e8b408eb8b2f3dd75011f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:12.794736 25010 consensus_queue.cc:1035] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:12.794874 25007 consensus_queue.cc:1035] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:12.795508 25004 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: TxnStatusTablet state changed. Reason: New leader e35a223c4e8b408eb8b2f3dd75011f9f. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.795630 25004 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.796264 25005 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: TxnStatusTablet state changed. Reason: New leader e35a223c4e8b408eb8b2f3dd75011f9f. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.796380 25005 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.796648 25007 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.796747 25007 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:12.797266 25010 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.797358 25010 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P e35a223c4e8b408eb8b2f3dd75011f9f: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:12.797662 25005 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.797775 25005 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P ec0a3a6096014669b0606e89f6513027: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.798080 25004 tablet_replica.cc:439] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 } } peers { permanent_uuid: "e35a223c4e8b408eb8b2f3dd75011f9f" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35825 } } peers { permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 } } }
I20260811 10:07:12.798178 25004 tablet_replica.cc:442] T b3a681cfedff49d5bb41a2602f9fb9a3 P ba1a3aedc33445d88ce20bd8da37492a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:12.804807 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.802367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35825 (local address 127.22.177.2:45327)
0811 10:07:12.802542 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:12.802552 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:12.802618 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:12.802763 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:12.802764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:12.802766 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:12.802830 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.802831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.803450 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.803452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.804015 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:12.804016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.804377 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.804378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:12.804427 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.804448 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:12.804640 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:12.804642 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:07:12.804692 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:07:12.804733 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260811 10:07:12.804807 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.802441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:45327 (local address 127.22.177.2:35825)
0811 10:07:12.802536 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.802538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.802624 (+ 86us) server_negotiation.cc:407] Connection header received
0811 10:07:12.802666 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.802667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.802677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.802758 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:12.802894 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.803359 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.804100 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.804290 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.804375 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.804547 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:12.804581 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:12.804639 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.804700 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.804716 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":77}
I20260811 10:07:12.807391 24922 raft_consensus.cc:1261] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Refusing update from remote peer e35a223c4e8b408eb8b2f3dd75011f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:12.807545 24612 raft_consensus.cc:1261] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Refusing update from remote peer e35a223c4e8b408eb8b2f3dd75011f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:12.807652 25007 consensus_queue.cc:1035] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec0a3a6096014669b0606e89f6513027" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:12.807771 25010 consensus_queue.cc:1035] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba1a3aedc33445d88ce20bd8da37492a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:12.809381 25013 mvcc.cc:205] Tried to move back new op lower bound from 7317269843177136128 to 7317269842508537856. Current Snapshot: MvccSnapshot[applied={T|T < 7317269843177136128}]
I20260811 10:07:12.823392 24435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:12.823498 24435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:12.824311 24435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c915c0124d9d47f1b9faeaded1b36e06: Sending DeleteTablet for 3 replicas of tablet 463db5beb904484bb21cd87d5d659dca
I20260811 10:07:12.824692 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet 463db5beb904484bb21cd87d5d659dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:12 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:51240
I20260811 10:07:12.824749 24901 tablet_service.cc:1514] Processing DeleteTablet for tablet 463db5beb904484bb21cd87d5d659dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:12 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:45128
I20260811 10:07:12.824865 25022 tablet_replica.cc:330] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: stopping tablet replica
I20260811 10:07:12.824862 25024 tablet_replica.cc:330] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: stopping tablet replica
I20260811 10:07:12.824918 25022 raft_consensus.cc:2229] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:12.824923 25024 raft_consensus.cc:2229] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:12.824961 25022 raft_consensus.cc:2258] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:12.824972 25024 raft_consensus.cc:2258] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:12.825052 24746 tablet_service.cc:1514] Processing DeleteTablet for tablet 463db5beb904484bb21cd87d5d659dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:12 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:57590
I20260811 10:07:12.825150 25026 tablet_replica.cc:330] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: stopping tablet replica
I20260811 10:07:12.825206 25026 raft_consensus.cc:2229] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:12.825274 25026 raft_consensus.cc:2258] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:12.825381 25024 ts_tablet_manager.cc:1905] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:12.825577 25026 ts_tablet_manager.cc:1905] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:12.827288 25026 ts_tablet_manager.cc:1918] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:12.827296 25024 ts_tablet_manager.cc:1918] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:12.827358 25026 log.cc:1196] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/463db5beb904484bb21cd87d5d659dca
I20260811 10:07:12.827356 25024 log.cc:1196] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/463db5beb904484bb21cd87d5d659dca
I20260811 10:07:12.827617 25026 ts_tablet_manager.cc:1939] T 463db5beb904484bb21cd87d5d659dca P e35a223c4e8b408eb8b2f3dd75011f9f: Deleting consensus metadata
I20260811 10:07:12.827620 25024 ts_tablet_manager.cc:1939] T 463db5beb904484bb21cd87d5d659dca P ba1a3aedc33445d88ce20bd8da37492a: Deleting consensus metadata
I20260811 10:07:12.828055 24421 catalog_manager.cc:4841] TS ba1a3aedc33445d88ce20bd8da37492a (127.22.177.1:43275): tablet 463db5beb904484bb21cd87d5d659dca (table test-table [id=b391bc9cb3394106b60c95df35b70ec3]) successfully deleted
I20260811 10:07:12.828750 24421 catalog_manager.cc:4841] TS e35a223c4e8b408eb8b2f3dd75011f9f (127.22.177.2:35825): tablet 463db5beb904484bb21cd87d5d659dca (table test-table [id=b391bc9cb3394106b60c95df35b70ec3]) successfully deleted
I20260811 10:07:12.829133 25022 ts_tablet_manager.cc:1905] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:12.831230 25022 ts_tablet_manager.cc:1918] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:12.831300 25022 log.cc:1196] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/463db5beb904484bb21cd87d5d659dca
I20260811 10:07:12.831524 25022 ts_tablet_manager.cc:1939] T 463db5beb904484bb21cd87d5d659dca P ec0a3a6096014669b0606e89f6513027: Deleting consensus metadata
I20260811 10:07:12.832235 24421 catalog_manager.cc:4841] TS ec0a3a6096014669b0606e89f6513027 (127.22.177.3:41637): tablet 463db5beb904484bb21cd87d5d659dca (table test-table [id=b391bc9cb3394106b60c95df35b70ec3]) successfully deleted
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[24391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:12.840070 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.828675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51244 (local address 127.22.177.1:43275)
0811 10:07:12.828735 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.828738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.828869 (+ 131us) server_negotiation.cc:407] Connection header received
0811 10:07:12.828976 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.828977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.828989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.829021 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:12.829931 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.830674 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.831654 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.831965 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.832046 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.835944 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:12.835960 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:12.835961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:12.835971 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:12.838519 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.838941 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.838946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.838946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.839026 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.839258 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.839259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.839259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.839467 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:12.839643 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.839835 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.839942 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 10:07:12.840545 24655 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51244
I20260811 10:07:12.849078 25038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:12.841825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.22.177.62:35497)
0811 10:07:12.841884 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:12.841886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:12.841898 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:12.841966 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:12.841967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:12.841979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:12.842058 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:12.842596 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.843334 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.844224 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:12.844454 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:12.844548 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:12.846045 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:12.846047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:12.846047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:12.846056 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:12.847698 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.848143 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.848144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.848145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.848209 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:12.848467 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:12.848468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:12.848469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:12.848602 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:12.848721 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:12.848833 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 10:07:12.848932 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 10:07:12.849344 24435 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50470
I20260811 10:07:12.850226 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 24518
I20260811 10:07:12.859563 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 24673
I20260811 10:07:12.869220 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 24825
I20260811 10:07:12.876816 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 24404
2026-08-11T10:07:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2878 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[25061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[25061](info): set up 2 sockets
Aug 11 10:07:12 dist-test-slave-g350 krb5kdc[25061](info): commencing operation
krb5kdc: starting...
W20260811 10:07:14.008564 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:14.159770 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45273
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:45273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:14.250356 25074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:14.250633 25074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:14.250677 25074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:14.252012 25074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:14.252104 25074 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 10:07:14.252128 25074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:14.252151 25074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:14.252182 25074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:14.252211 25074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:14.254522 25074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:45273
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:14.254931 25074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:14.255326 25074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:14.261946 25082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:14.261957 25085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:14.261997 25074 server_base.cc:1034] running on GCE node
W20260811 10:07:14.261957 25083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:14.262769 25074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:14.263192 25074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:14.264406 25074 hybrid_clock.cc:648] HybridClock initialized: now 1786442834264376 us; error 56 us; skew 500 ppm
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:14.267606 25074 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:14.268249 25074 webserver.cc:460] Webserver started at http://127.22.177.62:38699/ using document root <none> and password file <none>
I20260811 10:07:14.268508 25074 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:14.268582 25074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:14.268754 25074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:14.270087 25074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "0eb8645a94444cf4b791fff920fe9662"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "50f13d45e6ae525f0a04436c4484f27e"
server_key_iv: "40114345d4268748787a2360a9009b00"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.270491 25074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "0eb8645a94444cf4b791fff920fe9662"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "50f13d45e6ae525f0a04436c4484f27e"
server_key_iv: "40114345d4268748787a2360a9009b00"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.272089 25074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:14.273123 25092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.273437 25074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:14.273564 25074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "0eb8645a94444cf4b791fff920fe9662"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "50f13d45e6ae525f0a04436c4484f27e"
server_key_iv: "40114345d4268748787a2360a9009b00"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.273706 25074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:14.293424 25074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:14.296267 25074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:14.296568 25074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:14.302268 25074 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:45273
I20260811 10:07:14.302464 25161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:45273 every 8 connection(s)
I20260811 10:07:14.302747 25074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:14.303567 25162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:14.306143 25162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Bootstrap starting.
I20260811 10:07:14.306857 25162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:14.306862 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 25074
I20260811 10:07:14.306952 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:14.307148 23236 external_mini_cluster.cc:1428] Setting key 7adb176fcc847875202e69466eaed854
I20260811 10:07:14.307235 25162 log.cc:824] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:14.308051 25162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: No bootstrap required, opened a new log
I20260811 10:07:14.310412 25162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb8645a94444cf4b791fff920fe9662" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45273 } }
I20260811 10:07:14.310606 25162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:14.310645 25162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eb8645a94444cf4b791fff920fe9662, State: Initialized, Role: FOLLOWER
I20260811 10:07:14.310796 25162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb8645a94444cf4b791fff920fe9662" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45273 } }
I20260811 10:07:14.310895 25162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:14.310937 25162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:14.310994 25162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:14.311803 25162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb8645a94444cf4b791fff920fe9662" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45273 } }
I20260811 10:07:14.311908 25162 leader_election.cc:302] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0eb8645a94444cf4b791fff920fe9662; no voters:
I20260811 10:07:14.312158 25162 leader_election.cc:288] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:14.312218 25168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:14.312434 25168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [term 1 LEADER]: Becoming Leader. State: Replica: 0eb8645a94444cf4b791fff920fe9662, State: Running, Role: LEADER
I20260811 10:07:14.312465 25162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:14.312592 25168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb8645a94444cf4b791fff920fe9662" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45273 } }
I20260811 10:07:14.313077 25169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eb8645a94444cf4b791fff920fe9662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb8645a94444cf4b791fff920fe9662" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45273 } } }
I20260811 10:07:14.313166 25169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:14.313403 25170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eb8645a94444cf4b791fff920fe9662. Latest consensus state: current_term: 1 leader_uuid: "0eb8645a94444cf4b791fff920fe9662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb8645a94444cf4b791fff920fe9662" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45273 } } }
I20260811 10:07:14.313746 25170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:14.313951 25178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:14.314834 25178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:14.316730 25178 catalog_manager.cc:1347] Generated new cluster ID: cefb73afb11d4b57896e716a38a6b4c6
I20260811 10:07:14.316792 25178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:14.321383 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.307746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41398 (local address 127.22.177.62:45273)
0811 10:07:14.307934 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.307945 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.309838 (+ 1893us) server_negotiation.cc:407] Connection header received
0811 10:07:14.310395 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.310402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.310460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.310591 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.311469 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.312824 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.314098 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.314682 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.314812 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.317723 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.317728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.317728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.317742 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.319600 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.320094 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.320097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.320098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.320213 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.320448 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.320449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.320450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.320714 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.320900 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.321014 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.321170 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":136,"threads_started":1}
I20260811 10:07:14.346130 25178 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:14.346747 25178 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:14.351599 25178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Generated new TSK 0
I20260811 10:07:14.351863 25178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:14.389091 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45273
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:14.499528 25193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:14.499851 25193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:14.499888 25193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:14.501412 25193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:14.501490 25193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:14.501520 25193 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 10:07:14.501534 25193 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:14.501595 25193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:14.503664 25193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:45273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:14.504022 25193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:14.504319 25193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:14.510401 25199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:14.510489 25198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:14.510407 25201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:14.510893 25193 server_base.cc:1034] running on GCE node
I20260811 10:07:14.511206 25193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:14.511504 25193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:14.512761 25193 hybrid_clock.cc:648] HybridClock initialized: now 1786442834512721 us; error 55 us; skew 500 ppm
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:14.515887 25193 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:14.516566 25193 webserver.cc:460] Webserver started at http://127.22.177.1:38373/ using document root <none> and password file <none>
I20260811 10:07:14.516824 25193 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:14.516897 25193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:14.517055 25193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:14.518646 25193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "e3819229bb2a47009f1a40d2a563512b"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "6956d877d6dfc3f384c6ca865910a5b3"
server_key_iv: "893862fc2f907cc79ccfc82ee76ed017"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.519146 25193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "e3819229bb2a47009f1a40d2a563512b"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "6956d877d6dfc3f384c6ca865910a5b3"
server_key_iv: "893862fc2f907cc79ccfc82ee76ed017"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.521260 25193 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:07:14.522306 25209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.522521 25193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:14.522646 25193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "e3819229bb2a47009f1a40d2a563512b"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "6956d877d6dfc3f384c6ca865910a5b3"
server_key_iv: "893862fc2f907cc79ccfc82ee76ed017"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.522734 25193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:14.539893 25193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:14.542295 25193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:14.542491 25193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:14.543124 25193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:14.543192 25193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.543241 25193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:14.543295 25193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:14.552161 25193 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:39689
I20260811 10:07:14.552234 25333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:39689 every 8 connection(s)
I20260811 10:07:14.552802 25193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:14.555950 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.543894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:45931 (local address 127.22.177.62:45273)
0811 10:07:14.543955 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.543958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.544276 (+ 318us) server_negotiation.cc:407] Connection header received
0811 10:07:14.544823 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.544825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.544846 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.544939 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.545894 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.546783 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.548201 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.548589 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.548695 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.552289 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.552292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.552292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.552304 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.554101 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.554794 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.554796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.554796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.554871 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.555131 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.555132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.555132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.555332 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.555393 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.555768 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.555877 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:07:14.556073 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.543803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.22.177.1:45931)
0811 10:07:14.544082 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.544130 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.544705 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.545027 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.545031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.545458 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:14.545787 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.545794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.546913 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.546915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.548109 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.548113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.548715 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.548716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.548903 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.549633 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:14.549646 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:14.552135 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:14.554389 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.554397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.554400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.554661 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.554948 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.554949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.554950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.555066 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.555435 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:14.555443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:14.555584 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.555705 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.555888 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":173,"threads_started":1}
I20260811 10:07:14.556778 25335 heartbeater.cc:347] Connected to a master server at 127.22.177.62:45273
I20260811 10:07:14.556898 25335 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:14.557111 25335 heartbeater.cc:511] Master 127.22.177.62:45273 requested a full tablet report, sending...
I20260811 10:07:14.557449 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 25193
I20260811 10:07:14.557555 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:14.557770 25111 ts_manager.cc:194] Registered new tserver with Master: e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689)
I20260811 10:07:14.557791 23236 external_mini_cluster.cc:1428] Setting key 437cf25dfcf5e9d9aeece0ac733a8f99
I20260811 10:07:14.558746 25111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:45931
I20260811 10:07:14.564604 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.558006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.22.177.1:36651)
0811 10:07:14.558169 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.558183 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.558240 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.558379 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.558380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.558581 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:14.558647 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.558648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.559615 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.559617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.560142 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.560145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.560638 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.560639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.560752 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.561316 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:14.561321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:14.561730 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:14.563474 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.563475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.563475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.563650 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.563888 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.563889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.563894 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.563949 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.564225 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:14.564226 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:14.564286 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.564368 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.564492 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260811 10:07:14.564601 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.558193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:36651 (local address 127.22.177.62:45273)
0811 10:07:14.558233 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.558236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.558245 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:14.558304 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.558305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.558314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.558371 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.558732 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.559460 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.560231 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.560551 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.560634 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.561835 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.561837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.561837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.561845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.563355 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.563733 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.563735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.563735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.563797 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.564025 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.564026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.564026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.564159 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.564235 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.564394 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.564494 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:14.590420 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45273
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:14.681063 25345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:14.681313 25345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:14.681344 25345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:14.682664 25345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:14.682715 25345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:14.682739 25345 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 10:07:14.682750 25345 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:14.682770 25345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:14.684409 25345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:45273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:14.684731 25345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:14.685009 25345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:14.691058 25352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:14.691066 25350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:14.691073 25355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:14.691552 25345 server_base.cc:1034] running on GCE node
I20260811 10:07:14.691809 25345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:14.692045 25345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:14.693305 25345 hybrid_clock.cc:648] HybridClock initialized: now 1786442834693283 us; error 30 us; skew 500 ppm
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:14.696378 25345 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:14.696973 25345 webserver.cc:460] Webserver started at http://127.22.177.2:46649/ using document root <none> and password file <none>
I20260811 10:07:14.697247 25345 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:14.697297 25345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:14.697436 25345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:14.698648 25345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "947ab3414cc14fbc942d650f9f9af5bb"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "07edd91bea696c6d1b86911c45ce9d33"
server_key_iv: "01d12f38587c5aa32f9e131a56c371a1"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.699070 25345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "947ab3414cc14fbc942d650f9f9af5bb"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "07edd91bea696c6d1b86911c45ce9d33"
server_key_iv: "01d12f38587c5aa32f9e131a56c371a1"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.700631 25345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:14.701563 25362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.701833 25345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:14.701977 25345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "947ab3414cc14fbc942d650f9f9af5bb"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "07edd91bea696c6d1b86911c45ce9d33"
server_key_iv: "01d12f38587c5aa32f9e131a56c371a1"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.702092 25345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:14.712504 25345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:14.715238 25345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:14.715443 25345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:14.716104 25345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:14.716176 25345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.716223 25345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:14.716252 25345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.723984 25345 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35421
I20260811 10:07:14.724107 25486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35421 every 8 connection(s)
I20260811 10:07:14.724426 25345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:14.727403 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 25345
I20260811 10:07:14.727545 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:14.727783 23236 external_mini_cluster.cc:1428] Setting key 2dc7f331c043464731acbb366fe4b719
I20260811 10:07:14.729017 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.716848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:53719 (local address 127.22.177.62:45273)
0811 10:07:14.716902 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.716906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.717336 (+ 430us) server_negotiation.cc:407] Connection header received
0811 10:07:14.718060 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.718062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.718078 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.718136 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.719161 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.720053 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.721889 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.722233 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.722325 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.725363 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.725366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.725366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.725375 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.727157 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.727679 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.727681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.727681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.727761 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.728168 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.728169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.728170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.728317 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.728393 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.728770 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.728882 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:07:14.729053 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.716893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.22.177.2:53719)
0811 10:07:14.717123 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.717235 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.717926 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.718225 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.718232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.718664 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:14.719029 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.719038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.720183 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.720184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.721782 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.721786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.722371 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.722372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.722539 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.723289 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:14.723309 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:14.725238 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:14.727266 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.727276 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.727279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.727576 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.727892 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.727894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.727894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.728086 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.728402 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:14.728410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:14.728604 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.728722 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.728873 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":125,"threads_started":1}
I20260811 10:07:14.729579 25487 heartbeater.cc:347] Connected to a master server at 127.22.177.62:45273
I20260811 10:07:14.729694 25487 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:14.729907 25487 heartbeater.cc:511] Master 127.22.177.62:45273 requested a full tablet report, sending...
I20260811 10:07:14.730340 25111 ts_manager.cc:194] Registered new tserver with Master: 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421)
I20260811 10:07:14.730798 25111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:53719
I20260811 10:07:14.738205 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.731094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.22.177.2:43699)
0811 10:07:14.731268 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.731278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.731351 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.731466 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.731466 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.731675 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:14.731736 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.731737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.732654 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.732655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.733163 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.733165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.733765 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.733766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.733864 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.734465 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:14.734471 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:14.735006 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:14.736776 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.736777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.736777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.737019 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.737237 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.737238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.737238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.737319 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.737791 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:14.737793 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:14.737880 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.737964 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.738093 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260811 10:07:14.738148 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.731132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:43699 (local address 127.22.177.62:45273)
0811 10:07:14.731174 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.731176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.731337 (+ 161us) server_negotiation.cc:407] Connection header received
0811 10:07:14.731401 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.731402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.731413 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.731463 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.731823 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.732547 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.733255 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.733614 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.733689 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.735105 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.735106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.735107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.735113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.736673 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.737112 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.737113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.737114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.737169 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.737420 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.737421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.737421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.737682 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.737769 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.737993 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.738077 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:14.762070 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45273
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:14.861675 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:14.861938 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:14.861970 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:14.863389 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:14.863446 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:14.863471 25496 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 10:07:14.863485 25496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:14.863549 25496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:14.865314 25496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:45273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:14.865710 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:14.866020 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:14.871950 25505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:14.872045 25496 server_base.cc:1034] running on GCE node
W20260811 10:07:14.871971 25501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:14.871973 25503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:14.872622 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:14.872979 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:14.874191 25496 hybrid_clock.cc:648] HybridClock initialized: now 1786442834874154 us; error 45 us; skew 500 ppm
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:14.877135 25496 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:14.877748 25496 webserver.cc:460] Webserver started at http://127.22.177.3:39469/ using document root <none> and password file <none>
I20260811 10:07:14.878016 25496 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:14.878070 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:14.878212 25496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:14.879410 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "23b090f894854cd38fc136c6523ffebd"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "2fd44eb879f6f5aa3e42f2547b9917cd"
server_key_iv: "6c45e4c32096790c912ee4ff804fa35a"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.879808 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "23b090f894854cd38fc136c6523ffebd"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "2fd44eb879f6f5aa3e42f2547b9917cd"
server_key_iv: "6c45e4c32096790c912ee4ff804fa35a"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.881382 25496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:14.882529 25513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.882912 25496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:14.883028 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "23b090f894854cd38fc136c6523ffebd"
format_stamp: "Formatted at 2026-08-11 10:07:14 on dist-test-slave-g350"
server_key: "2fd44eb879f6f5aa3e42f2547b9917cd"
server_key_iv: "6c45e4c32096790c912ee4ff804fa35a"
server_key_version: "encryptionkey@0"
I20260811 10:07:14.883113 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:14.897073 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:14.899713 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:14.899889 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:14.900466 25496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:14.900506 25496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.900530 25496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:14.900544 25496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:14.908002 25496 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:34979
I20260811 10:07:14.908104 25637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:34979 every 8 connection(s)
I20260811 10:07:14.908485 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:14.909122 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 25496
I20260811 10:07:14.909231 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:14.909474 23236 external_mini_cluster.cc:1428] Setting key 05fe649253dcdf801468d87e51b33de7
I20260811 10:07:14.913210 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.901145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:42647 (local address 127.22.177.62:45273)
0811 10:07:14.901220 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.901223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.901519 (+ 296us) server_negotiation.cc:407] Connection header received
0811 10:07:14.902283 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.902285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.902301 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.902387 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.903392 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.904251 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.906100 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.906474 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.906566 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.909766 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.909769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.909769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.909780 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.911577 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.912052 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.912053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.912053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.912115 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.912415 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.912416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.912416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.912543 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.912600 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.912993 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.913136 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:07:14.913393 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.901103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.22.177.3:42647)
0811 10:07:14.901315 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.901414 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.902149 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.902497 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.902503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.902938 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:14.903272 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.903279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.904444 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.904445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.905993 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.905997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.906588 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.906590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.906793 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.907538 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:14.907548 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:14.909640 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:14.911689 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.911699 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.911702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.911973 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.912205 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.912206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.912207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.912327 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.912642 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:14.912649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:14.912789 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.912950 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.913160 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":131,"threads_started":1}
I20260811 10:07:14.913926 25638 heartbeater.cc:347] Connected to a master server at 127.22.177.62:45273
I20260811 10:07:14.914027 25638 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:14.914278 25638 heartbeater.cc:511] Master 127.22.177.62:45273 requested a full tablet report, sending...
I20260811 10:07:14.914708 25110 ts_manager.cc:194] Registered new tserver with Master: 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:14.915144 25110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:42647
I20260811 10:07:14.921676 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:07:14.921717 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.915311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:54493 (local address 127.22.177.62:45273)
0811 10:07:14.915365 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.915368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.915547 (+ 179us) server_negotiation.cc:407] Connection header received
0811 10:07:14.915605 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.915606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.915615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.915661 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.916110 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.916615 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.917506 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.917747 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.917806 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.918861 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.918862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.918862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.918870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.920310 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.920703 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.920704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.920704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.920765 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.921007 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.921009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.921009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.921172 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.921277 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.921469 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.921569 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:07:14.921758 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.915254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.22.177.3:54493)
0811 10:07:14.915439 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.915449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.915542 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.915668 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.915669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.915912 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:14.915993 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.915995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.916714 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.916715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.917428 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.917430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.917826 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.917827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.917899 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.918392 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:14.918397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:14.918749 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:14.920410 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.920412 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.920412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.920606 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.920843 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:14.920845 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:14.920845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:14.920922 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:14.921280 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:14.921281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:14.921361 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.921439 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.921571 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:07:14 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:14.950968 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.941941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.22.177.62:45273)
0811 10:07:14.941994 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.941998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.942040 (+ 42us) server_negotiation.cc:407] Connection header received
0811 10:07:14.942107 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.942109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.942119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.942173 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:14.942963 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.943548 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.944421 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.944798 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.944879 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.947912 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:14.947914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:14.947914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:14.947924 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:14.949573 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.950054 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.950056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.950056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.950197 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:14.950445 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:14.950447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:14.950447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:14.950601 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:14.950702 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.950821 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.950895 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:07:14.953243 25110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:14.955140 25110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:14.962813 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.958287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:39689 (local address 127.0.0.1:39562)
0811 10:07:14.958514 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.958559 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.958640 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.959228 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.959229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.959232 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:14.959306 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.959308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.960269 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.960270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.961663 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.961666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.962447 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.962448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.962533 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.962547 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.962625 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.962704 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260811 10:07:14.962941 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.958288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:34979 (local address 127.0.0.1:56804)
0811 10:07:14.958758 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.958775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.958832 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.959077 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.959081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.959083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:14.959161 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.959164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.960579 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.960580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.961558 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.961561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.962616 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.962617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.962710 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.962715 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.962750 (+ 35us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.962814 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":23,"spinlock_wait_cycles":896,"thread_start_us":127,"threads_started":1}
I20260811 10:07:14.963053 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.958580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56804 (local address 127.22.177.3:34979)
0811 10:07:14.958772 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.958777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.958882 (+ 105us) server_negotiation.cc:407] Connection header received
0811 10:07:14.958931 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.958933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.958950 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.959255 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:14.959326 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.960472 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.961710 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.962482 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.962801 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.962865 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.962892 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.962932 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260811 10:07:14.963071 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.958288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.0.0.1:40192)
0811 10:07:14.958581 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:14.958591 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:14.958653 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:14.959134 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:14.959134 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:14.959136 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:14.959248 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.959250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.960548 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.960549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.961741 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:14.961743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.962618 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.962620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:14.962681 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.962684 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:14.962751 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:07:14.962860 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":4,"thread_start_us":131,"threads_started":1}
I20260811 10:07:14.963260 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.958409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.22.177.1:39689)
0811 10:07:14.958940 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.958943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.958953 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:14.959015 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.959017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.959034 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.959398 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:14.959465 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.960153 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.961775 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.962327 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.962444 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.962525 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.963066 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.963131 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":89,"threads_started":1}
I20260811 10:07:14.963994 25263 tablet_service.cc:1467] Processing CreateTablet for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 (DEFAULT_TABLE table=test-table [id=3199cd091cb847be90283dab6f311f9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:14.963999 25568 tablet_service.cc:1467] Processing CreateTablet for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 (DEFAULT_TABLE table=test-table [id=3199cd091cb847be90283dab6f311f9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:14.964395 25263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:14.964395 25568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:14.965978 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:14.958580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.22.177.2:35421)
0811 10:07:14.958880 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:14.958883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:14.958894 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:07:14.958931 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:14.958933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:14.958982 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:14.959059 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:14.959335 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.960448 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.961831 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:14.962530 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:14.962597 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:14.962659 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:14.962766 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 10:07:14.962810 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":97,"threads_started":1}
I20260811 10:07:14.966933 25415 tablet_service.cc:1467] Processing CreateTablet for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 (DEFAULT_TABLE table=test-table [id=3199cd091cb847be90283dab6f311f9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:14.967324 25415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:14.968096 25661 tablet_bootstrap.cc:492] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Bootstrap starting.
I20260811 10:07:14.968441 25662 tablet_bootstrap.cc:492] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Bootstrap starting.
I20260811 10:07:14.969039 25662 tablet_bootstrap.cc:654] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:14.969298 25661 tablet_bootstrap.cc:654] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:14.969424 25662 log.cc:824] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:14.969774 25661 log.cc:824] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:14.970839 25661 tablet_bootstrap.cc:492] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: No bootstrap required, opened a new log
I20260811 10:07:14.970928 25661 ts_tablet_manager.cc:1397] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:07:14.971764 25665 tablet_bootstrap.cc:492] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Bootstrap starting.
I20260811 10:07:14.972220 25662 tablet_bootstrap.cc:492] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: No bootstrap required, opened a new log
I20260811 10:07:14.972317 25662 ts_tablet_manager.cc:1397] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 10:07:14.972975 25661 raft_consensus.cc:348] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:14.973137 25661 raft_consensus.cc:374] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:14.973163 25661 raft_consensus.cc:729] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Initialized, Role: FOLLOWER
I20260811 10:07:14.973311 25661 consensus_queue.cc:260] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:14.973698 25638 heartbeater.cc:503] Master 127.22.177.62:45273 was elected leader, sending a full tablet report...
I20260811 10:07:14.974030 25661 ts_tablet_manager.cc:1428] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:14.974427 25665 tablet_bootstrap.cc:654] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:14.974370 25662 raft_consensus.cc:348] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:14.974522 25662 raft_consensus.cc:374] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:14.974565 25662 raft_consensus.cc:729] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3819229bb2a47009f1a40d2a563512b, State: Initialized, Role: FOLLOWER
I20260811 10:07:14.974699 25662 consensus_queue.cc:260] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:14.974851 25665 log.cc:824] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:14.975128 25662 ts_tablet_manager.cc:1428] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 10:07:14.975271 25335 heartbeater.cc:503] Master 127.22.177.62:45273 was elected leader, sending a full tablet report...
I20260811 10:07:14.976079 25665 tablet_bootstrap.cc:492] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: No bootstrap required, opened a new log
I20260811 10:07:14.976169 25665 ts_tablet_manager.cc:1397] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 10:07:14.978111 25665 raft_consensus.cc:348] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:14.978282 25665 raft_consensus.cc:374] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:14.978307 25665 raft_consensus.cc:729] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Initialized, Role: FOLLOWER
I20260811 10:07:14.978449 25665 consensus_queue.cc:260] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:14.978760 25665 ts_tablet_manager.cc:1428] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:14.978894 25487 heartbeater.cc:503] Master 127.22.177.62:45273 was elected leader, sending a full tablet report...
W20260811 10:07:15.053864 25336 tablet.cc:2369] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:15.078948 25666 raft_consensus.cc:482] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:15.079095 25666 raft_consensus.cc:504] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:15.079512 25666 leader_election.cc:288] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421)
I20260811 10:07:15.083881 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.079897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52799 (local address 127.22.177.2:35421)
0811 10:07:15.079944 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.079946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.080141 (+ 195us) server_negotiation.cc:407] Connection header received
0811 10:07:15.080170 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.080171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.080178 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.080261 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:15.080541 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.081229 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.082747 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.083332 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.083424 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.083436 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.083750 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.083804 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:07:15.083881 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.079856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:47241 (local address 127.22.177.1:39689)
0811 10:07:15.079906 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.079909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.079923 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:07:15.079962 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.079963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.079973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.080054 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:15.080236 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.081039 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.082748 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.083426 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.083501 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.083507 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.083753 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.083804 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:07:15.083868 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.079779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:39689 (local address 127.22.177.3:47241)
0811 10:07:15.079839 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.079851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.079900 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.080060 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.080061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.080063 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:15.080153 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.080155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.081208 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.081210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.082572 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.082575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.083521 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.083522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.083611 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.083613 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.083695 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.083744 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":38}
I20260811 10:07:15.083885 25670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.079788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.22.177.3:52799)
0811 10:07:15.080058 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.080068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.080125 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.080371 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.080372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.080374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:15.080450 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.080451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.081317 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.081318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.082572 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.082575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.083447 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.083448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.083569 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.083575 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.083704 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.083744 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":66,"thread_start_us":136,"threads_started":1}
I20260811 10:07:15.084393 25439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "589d27fa5ddf4c9da88cb0b6dd1b3b93" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ab3414cc14fbc942d650f9f9af5bb" is_pre_election: true
I20260811 10:07:15.084472 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "589d27fa5ddf4c9da88cb0b6dd1b3b93" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b" is_pre_election: true
I20260811 10:07:15.084621 25439 raft_consensus.cc:2454] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 0.
I20260811 10:07:15.084652 25285 raft_consensus.cc:2454] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 0.
I20260811 10:07:15.084918 25517 leader_election.cc:302] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b090f894854cd38fc136c6523ffebd, 947ab3414cc14fbc942d650f9f9af5bb; no voters:
I20260811 10:07:15.085076 25666 raft_consensus.cc:2781] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:15.085139 25666 raft_consensus.cc:482] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:15.085175 25666 raft_consensus.cc:3037] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.086004 25666 raft_consensus.cc:504] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:15.086190 25666 leader_election.cc:288] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 election: Requested vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421)
I20260811 10:07:15.086400 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "589d27fa5ddf4c9da88cb0b6dd1b3b93" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b"
I20260811 10:07:15.086447 25439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "589d27fa5ddf4c9da88cb0b6dd1b3b93" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ab3414cc14fbc942d650f9f9af5bb"
I20260811 10:07:15.086494 25285 raft_consensus.cc:3037] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.086565 25439 raft_consensus.cc:3037] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.087289 25285 raft_consensus.cc:2454] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 1.
I20260811 10:07:15.087741 25439 raft_consensus.cc:2454] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 1.
I20260811 10:07:15.088052 25518 leader_election.cc:302] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b090f894854cd38fc136c6523ffebd, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:15.088171 25666 raft_consensus.cc:2781] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:15.088335 25666 raft_consensus.cc:686] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 1 LEADER]: Becoming Leader. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Running, Role: LEADER
I20260811 10:07:15.088450 25666 consensus_queue.cc:237] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } }
I20260811 10:07:15.089483 25110 catalog_manager.cc:5495] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b090f894854cd38fc136c6523ffebd (127.22.177.3). New cstate: current_term: 1 leader_uuid: "23b090f894854cd38fc136c6523ffebd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } health_report { overall_health: UNKNOWN } } }
Aug 11 10:07:15 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:15.131343 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.122632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45273 (local address 127.0.0.1:41414)
0811 10:07:15.122865 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.122890 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.122957 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.123146 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.123147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.123505 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:15.123557 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.123559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.124325 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.124326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.124857 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.124858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.125221 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.125222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.125285 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.126006 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:15.126014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:15.127825 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:15.129883 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:15.129894 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:15.129896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:15.130144 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:15.130566 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:15.130567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:15.130567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:15.130690 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:15.130971 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:15.130980 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:15.131044 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.131123 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.131259 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":155,"threads_started":1}
I20260811 10:07:15.131409 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.122704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.22.177.62:45273)
0811 10:07:15.122812 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.122814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.122979 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:07:15.123012 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.123013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.123026 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.123099 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:15.123657 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.124170 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.124945 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.125130 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.125190 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.127940 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:15.127942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:15.127942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:15.127950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:15.129781 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:15.130340 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:15.130341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:15.130342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:15.130461 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:15.130766 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:15.130766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:15.130767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:15.130902 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:15.130961 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.131189 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.131315 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":71}
I20260811 10:07:15.134047 25110 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41414:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 10:07:15.134295 25110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:15.136052 25415 tablet_service.cc:1467] Processing CreateTablet for tablet 30a8a32cdc1642f09fd6889ab27a50f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=953b2e4e16684c0eb4270db838171c3d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:15.136044 25568 tablet_service.cc:1467] Processing CreateTablet for tablet 30a8a32cdc1642f09fd6889ab27a50f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=953b2e4e16684c0eb4270db838171c3d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:15.136037 25263 tablet_service.cc:1467] Processing CreateTablet for tablet 30a8a32cdc1642f09fd6889ab27a50f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=953b2e4e16684c0eb4270db838171c3d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:15.136250 25263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30a8a32cdc1642f09fd6889ab27a50f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:15.136250 25568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30a8a32cdc1642f09fd6889ab27a50f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:15.136255 25415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30a8a32cdc1642f09fd6889ab27a50f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:15.137660 25665 tablet_bootstrap.cc:492] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: Bootstrap starting.
I20260811 10:07:15.137858 25662 tablet_bootstrap.cc:492] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: Bootstrap starting.
I20260811 10:07:15.137858 25661 tablet_bootstrap.cc:492] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: Bootstrap starting.
I20260811 10:07:15.138315 25661 tablet_bootstrap.cc:654] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:15.138321 25662 tablet_bootstrap.cc:654] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:15.138463 25665 tablet_bootstrap.cc:654] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:15.139125 25662 tablet_bootstrap.cc:492] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: No bootstrap required, opened a new log
I20260811 10:07:15.139211 25662 ts_tablet_manager.cc:1397] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260811 10:07:15.139384 25661 tablet_bootstrap.cc:492] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: No bootstrap required, opened a new log
I20260811 10:07:15.139448 25661 ts_tablet_manager.cc:1397] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:15.139394 25662 raft_consensus.cc:348] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.139508 25662 raft_consensus.cc:374] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:15.139536 25662 raft_consensus.cc:729] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3819229bb2a47009f1a40d2a563512b, State: Initialized, Role: FOLLOWER
I20260811 10:07:15.139603 25665 tablet_bootstrap.cc:492] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: No bootstrap required, opened a new log
I20260811 10:07:15.139595 25662 consensus_queue.cc:260] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.139647 25665 ts_tablet_manager.cc:1397] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:15.139643 25661 raft_consensus.cc:348] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.139706 25662 ts_tablet_manager.cc:1428] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:15.139735 25661 raft_consensus.cc:374] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:15.139758 25661 raft_consensus.cc:729] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Initialized, Role: FOLLOWER
I20260811 10:07:15.139814 25661 consensus_queue.cc:260] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.139796 25667 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.139837 25665 raft_consensus.cc:348] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.139897 25665 raft_consensus.cc:374] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:15.139912 25665 raft_consensus.cc:729] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Initialized, Role: FOLLOWER
I20260811 10:07:15.139889 25667 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:15.139905 25661 ts_tablet_manager.cc:1428] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:15.139976 25665 consensus_queue.cc:260] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.140084 25665 ts_tablet_manager.cc:1428] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:15.140138 25666 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.140209 25666 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:15.140246 25669 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.140353 25669 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 10:07:15.159192 25639 tablet.cc:2369] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:07:15.225342 25488 tablet.cc:2369] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:15.250721 25669 raft_consensus.cc:482] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:15.250840 25669 raft_consensus.cc:504] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.251294 25669 leader_election.cc:288] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:15.254387 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.251577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:43389 (local address 127.22.177.3:34979)
0811 10:07:15.251691 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.251694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.251712 (+ 18us) server_negotiation.cc:407] Connection header received
0811 10:07:15.251801 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.251802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.251812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.251907 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:15.252066 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.252549 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.253567 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.254007 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.254101 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.254111 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.254262 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.254305 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 10:07:15.254398 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.251514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:34979 (local address 127.22.177.2:43389)
0811 10:07:15.251649 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.251663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.251733 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.251918 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.251919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.251922 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:15.251987 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.251988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.252676 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.252677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.253455 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.253457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.254101 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.254102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.254182 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.254185 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.254256 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.254306 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260811 10:07:15.254784 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.251694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:40615 (local address 127.22.177.1:39689)
0811 10:07:15.251742 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.251745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.251892 (+ 147us) server_negotiation.cc:407] Connection header received
0811 10:07:15.252020 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.252021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.252030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.252100 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:15.252338 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.252881 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.253955 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.254357 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.254457 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.254465 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.254664 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.254717 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:07:15.254786 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a8a32cdc1642f09fd6889ab27a50f2" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b090f894854cd38fc136c6523ffebd" is_pre_election: true
I20260811 10:07:15.254928 25589 raft_consensus.cc:2454] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 0.
I20260811 10:07:15.255051 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.251533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:39689 (local address 127.22.177.2:40615)
0811 10:07:15.251793 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.251805 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.251908 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.252105 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.252106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.252108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:15.252194 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.252195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.252982 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.252983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.253855 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.253857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.254457 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.254457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.254549 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.254555 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.254908 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.254969 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":78,"threads_started":1}
I20260811 10:07:15.255170 25364 leader_election.cc:302] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b090f894854cd38fc136c6523ffebd, 947ab3414cc14fbc942d650f9f9af5bb; no voters:
I20260811 10:07:15.255360 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a8a32cdc1642f09fd6889ab27a50f2" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b" is_pre_election: true
I20260811 10:07:15.255432 25669 raft_consensus.cc:2781] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:15.255478 25669 raft_consensus.cc:482] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:15.255479 25285 raft_consensus.cc:2454] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 0.
I20260811 10:07:15.255504 25669 raft_consensus.cc:3037] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.256402 25669 raft_consensus.cc:504] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.256721 25669 leader_election.cc:288] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 election: Requested vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:15.256925 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a8a32cdc1642f09fd6889ab27a50f2" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b"
I20260811 10:07:15.257025 25285 raft_consensus.cc:3037] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.257077 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a8a32cdc1642f09fd6889ab27a50f2" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b090f894854cd38fc136c6523ffebd"
I20260811 10:07:15.257192 25589 raft_consensus.cc:3037] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.257969 25285 raft_consensus.cc:2454] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 1.
I20260811 10:07:15.258103 25589 raft_consensus.cc:2454] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 1.
I20260811 10:07:15.258176 25366 leader_election.cc:302] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947ab3414cc14fbc942d650f9f9af5bb, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:15.258327 25669 raft_consensus.cc:2781] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:15.258486 25669 raft_consensus.cc:686] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 LEADER]: Becoming Leader. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Running, Role: LEADER
I20260811 10:07:15.258625 25669 consensus_queue.cc:237] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.259162 25687 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: TxnStatusTablet state changed. Reason: New leader 947ab3414cc14fbc942d650f9f9af5bb. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.259295 25687 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:15.259501 25689 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 10:07:15.259622 25689 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 10:07:15.259635 25111 catalog_manager.cc:5495] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2). New cstate: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:15.293134 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.289989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.0.0.1:40204)
0811 10:07:15.290102 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.290114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.290171 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.290395 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.290396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.290397 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:15.290446 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.290448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.291463 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.291464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.292033 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.292035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.292557 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.292558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.292611 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.292627 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.292966 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.292976 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.293019 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.293068 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 10:07:15.293139 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.290057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.22.177.2:35421)
0811 10:07:15.290133 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.290136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.290164 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:07:15.290253 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.290254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.290279 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.290356 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:15.290520 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.291366 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.292123 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.292468 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.292550 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.292731 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.292889 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.292982 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.293024 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.293049 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 10:07:15.295241 25589 raft_consensus.cc:1261] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Refusing update from remote peer 947ab3414cc14fbc942d650f9f9af5bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:15.295277 25285 raft_consensus.cc:1261] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Refusing update from remote peer 947ab3414cc14fbc942d650f9f9af5bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:15.295671 25687 consensus_queue.cc:1035] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:15.295846 25669 consensus_queue.cc:1035] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:15.297233 25666 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: TxnStatusTablet state changed. Reason: New leader 947ab3414cc14fbc942d650f9f9af5bb. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.297272 25667 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: TxnStatusTablet state changed. Reason: New leader 947ab3414cc14fbc942d650f9f9af5bb. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.297339 25666 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:15.297370 25667 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:15.298593 25669 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.298700 25669 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:15.298833 25666 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.298882 25692 mvcc.cc:205] Tried to move back new op lower bound from 7317269853366284288 to 7317269853220028416. Current Snapshot: MvccSnapshot[applied={T|T < 7317269853366284288}]
I20260811 10:07:15.298928 25666 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 23b090f894854cd38fc136c6523ffebd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:15.299325 25693 mvcc.cc:205] Tried to move back new op lower bound from 7317269853366284288 to 7317269853220028416. Current Snapshot: MvccSnapshot[applied={T|T < 7317269853366284288}]
I20260811 10:07:15.299878 25669 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.299969 25669 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P 947ab3414cc14fbc942d650f9f9af5bb: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:15.300523 25667 tablet_replica.cc:439] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } } }
I20260811 10:07:15.300624 25667 tablet_replica.cc:442] T 30a8a32cdc1642f09fd6889ab27a50f2 P e3819229bb2a47009f1a40d2a563512b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:15.308565 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.305631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.22.177.3:34979)
0811 10:07:15.305696 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.305698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.305710 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:15.305756 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.305757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.305775 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.305898 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:15.306069 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.306665 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.307680 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.307893 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.307975 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.308174 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.308288 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.308402 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.308425 (+ 23us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.308449 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:07:15.314755 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.311801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.22.177.3:50077)
0811 10:07:15.311866 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.311880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.311946 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.312124 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.312125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.312126 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:15.312206 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.312207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.313038 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.313040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.313837 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.313840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.314229 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.314230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.314317 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.314335 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.314581 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.314583 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.314635 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.314686 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 10:07:15.314788 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.311878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:50077 (local address 127.22.177.2:35421)
0811 10:07:15.311967 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.311970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.311978 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:15.312008 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.312009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.312020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.312121 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:15.312361 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.312953 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.313942 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.314151 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.314230 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.314444 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.314489 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.314576 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.314671 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.314702 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 10:07:15.317888 25439 raft_consensus.cc:1261] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Refusing update from remote peer 23b090f894854cd38fc136c6523ffebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:15.317863 25285 raft_consensus.cc:1261] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Refusing update from remote peer 23b090f894854cd38fc136c6523ffebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:15.318173 25666 consensus_queue.cc:1035] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:15.318307 25671 consensus_queue.cc:1035] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:15.320758 25695 mvcc.cc:205] Tried to move back new op lower bound from 7317269853459394560 to 7317269852523122688. Current Snapshot: MvccSnapshot[applied={T|T < 7317269853459394560}]
I20260811 10:07:15.328884 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.325595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:34979 (local address 127.22.177.2:39547)
0811 10:07:15.325835 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:15.325853 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:15.325952 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:15.326098 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:15.326099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:15.326101 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:15.326182 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.326184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.326997 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.326999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.327950 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:15.327953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.328363 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.328364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:15.328436 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.328463 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.328694 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.328697 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:15.328758 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:07:15.328805 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260811 10:07:15.328881 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:15.325701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:39547 (local address 127.22.177.3:34979)
0811 10:07:15.325752 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:15.325755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:15.325923 (+ 168us) server_negotiation.cc:407] Connection header received
0811 10:07:15.326020 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:15.326021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:15.326033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:15.326094 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:15.326317 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.326905 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.328041 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:15.328264 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:15.328361 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:15.328576 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:15.328625 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:15.328694 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:15.328763 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 10:07:15.328789 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:07:15.343221 25111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:15.343376 25111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:15.344416 25111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Sending DeleteTablet for 3 replicas of tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93
I20260811 10:07:15.344828 25568 tablet_service.cc:1514] Processing DeleteTablet for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:15 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:56804
I20260811 10:07:15.344913 25415 tablet_service.cc:1514] Processing DeleteTablet for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:15 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:40192
I20260811 10:07:15.345374 25263 tablet_service.cc:1514] Processing DeleteTablet for tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:15 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:39562
I20260811 10:07:15.345639 25111 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 10:07:15.345728 25715 tablet_replica.cc:330] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: stopping tablet replica
W20260811 10:07:15.345839 25111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:15.346136 25715 raft_consensus.cc:2229] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:15.346287 25715 raft_consensus.cc:2258] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:15.347055 25714 tablet_replica.cc:330] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: stopping tablet replica
I20260811 10:07:15.347193 25714 raft_consensus.cc:2229] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:15.347261 25714 raft_consensus.cc:2258] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:15.347543 25713 tablet_replica.cc:330] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: stopping tablet replica
I20260811 10:07:15.347551 25263 tablet_service.cc:1467] Processing CreateTablet for tablet 3505748360954e0e85702d290a940475 (DEFAULT_TABLE table=test-table [id=47c9ac720c3a4851881ada7aba678588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:15.347671 25713 raft_consensus.cc:2229] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:15.347700 25263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3505748360954e0e85702d290a940475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:15.347810 25713 raft_consensus.cc:2258] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:15.348196 25714 ts_tablet_manager.cc:1905] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:15.349201 25662 tablet_bootstrap.cc:492] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Bootstrap starting.
I20260811 10:07:15.349316 25415 tablet_service.cc:1467] Processing CreateTablet for tablet 3505748360954e0e85702d290a940475 (DEFAULT_TABLE table=test-table [id=47c9ac720c3a4851881ada7aba678588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:15.349455 25415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3505748360954e0e85702d290a940475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:15.349680 25662 tablet_bootstrap.cc:654] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:15.349721 25568 tablet_service.cc:1467] Processing CreateTablet for tablet 3505748360954e0e85702d290a940475 (DEFAULT_TABLE table=test-table [id=47c9ac720c3a4851881ada7aba678588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:15.349840 25568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3505748360954e0e85702d290a940475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:15.351094 25665 tablet_bootstrap.cc:492] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Bootstrap starting.
I20260811 10:07:15.351230 25661 tablet_bootstrap.cc:492] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Bootstrap starting.
I20260811 10:07:15.351645 25665 tablet_bootstrap.cc:654] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:15.351758 25661 tablet_bootstrap.cc:654] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:15.352285 25715 ts_tablet_manager.cc:1905] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:15.352576 25713 ts_tablet_manager.cc:1905] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:15.353709 25714 ts_tablet_manager.cc:1918] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:15.353819 25714 log.cc:1196] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/589d27fa5ddf4c9da88cb0b6dd1b3b93
I20260811 10:07:15.354157 25714 ts_tablet_manager.cc:1939] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting consensus metadata
I20260811 10:07:15.354468 25715 ts_tablet_manager.cc:1918] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:15.354532 25713 ts_tablet_manager.cc:1918] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:15.354616 25713 log.cc:1196] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/589d27fa5ddf4c9da88cb0b6dd1b3b93
I20260811 10:07:15.354698 25715 log.cc:1196] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/589d27fa5ddf4c9da88cb0b6dd1b3b93
I20260811 10:07:15.354687 25095 catalog_manager.cc:4841] TS 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421): tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 (table test-table [id=3199cd091cb847be90283dab6f311f9a]) successfully deleted
I20260811 10:07:15.354960 25713 ts_tablet_manager.cc:1939] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P 23b090f894854cd38fc136c6523ffebd: Deleting consensus metadata
I20260811 10:07:15.355157 25715 ts_tablet_manager.cc:1939] T 589d27fa5ddf4c9da88cb0b6dd1b3b93 P e3819229bb2a47009f1a40d2a563512b: Deleting consensus metadata
I20260811 10:07:15.355480 25094 catalog_manager.cc:4841] TS 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979): tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 (table test-table [id=3199cd091cb847be90283dab6f311f9a]) successfully deleted
I20260811 10:07:15.355597 25665 tablet_bootstrap.cc:492] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: No bootstrap required, opened a new log
I20260811 10:07:15.355648 25665 ts_tablet_manager.cc:1397] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 10:07:15.355719 25096 catalog_manager.cc:4841] TS e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689): tablet 589d27fa5ddf4c9da88cb0b6dd1b3b93 (table test-table [id=3199cd091cb847be90283dab6f311f9a]) successfully deleted
I20260811 10:07:15.355820 25665 raft_consensus.cc:348] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.355931 25665 raft_consensus.cc:374] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:15.355959 25665 raft_consensus.cc:729] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Initialized, Role: FOLLOWER
I20260811 10:07:15.356005 25665 consensus_queue.cc:260] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.356101 25665 ts_tablet_manager.cc:1428] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:15.358161 25661 tablet_bootstrap.cc:492] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: No bootstrap required, opened a new log
I20260811 10:07:15.358234 25661 ts_tablet_manager.cc:1397] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260811 10:07:15.358429 25661 raft_consensus.cc:348] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.358526 25661 raft_consensus.cc:374] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:15.358551 25661 raft_consensus.cc:729] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Initialized, Role: FOLLOWER
I20260811 10:07:15.358598 25661 consensus_queue.cc:260] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.358695 25661 ts_tablet_manager.cc:1428] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:15.366286 25662 tablet_bootstrap.cc:492] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: No bootstrap required, opened a new log
I20260811 10:07:15.366385 25662 ts_tablet_manager.cc:1397] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20260811 10:07:15.366583 25662 raft_consensus.cc:348] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.366696 25662 raft_consensus.cc:374] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:15.366724 25662 raft_consensus.cc:729] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3819229bb2a47009f1a40d2a563512b, State: Initialized, Role: FOLLOWER
I20260811 10:07:15.366786 25662 consensus_queue.cc:260] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.366923 25662 ts_tablet_manager.cc:1428] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:15.412277 25708 raft_consensus.cc:482] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:15.412405 25708 raft_consensus.cc:504] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.412611 25708 leader_election.cc:288] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421)
I20260811 10:07:15.412909 25439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3505748360954e0e85702d290a940475" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ab3414cc14fbc942d650f9f9af5bb" is_pre_election: true
I20260811 10:07:15.413007 25439 raft_consensus.cc:2454] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 0.
I20260811 10:07:15.413019 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3505748360954e0e85702d290a940475" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b" is_pre_election: true
I20260811 10:07:15.413156 25285 raft_consensus.cc:2454] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 0.
I20260811 10:07:15.413278 25517 leader_election.cc:302] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b090f894854cd38fc136c6523ffebd, 947ab3414cc14fbc942d650f9f9af5bb; no voters:
I20260811 10:07:15.413457 25708 raft_consensus.cc:2781] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:15.413502 25708 raft_consensus.cc:482] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:15.413525 25708 raft_consensus.cc:3037] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.414577 25708 raft_consensus.cc:504] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.414737 25708 leader_election.cc:288] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 election: Requested vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421)
I20260811 10:07:15.414958 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3505748360954e0e85702d290a940475" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b"
I20260811 10:07:15.415043 25285 raft_consensus.cc:3037] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.415009 25439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3505748360954e0e85702d290a940475" candidate_uuid: "23b090f894854cd38fc136c6523ffebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ab3414cc14fbc942d650f9f9af5bb"
I20260811 10:07:15.415089 25439 raft_consensus.cc:3037] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:15.415889 25285 raft_consensus.cc:2454] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 1.
I20260811 10:07:15.415972 25439 raft_consensus.cc:2454] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b090f894854cd38fc136c6523ffebd in term 1.
I20260811 10:07:15.416069 25518 leader_election.cc:302] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b090f894854cd38fc136c6523ffebd, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:15.416270 25708 raft_consensus.cc:2781] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:15.416333 25708 raft_consensus.cc:686] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 1 LEADER]: Becoming Leader. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Running, Role: LEADER
I20260811 10:07:15.416393 25708 consensus_queue.cc:237] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:15.417002 25111 catalog_manager.cc:5495] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b090f894854cd38fc136c6523ffebd (127.22.177.3). New cstate: current_term: 1 leader_uuid: "23b090f894854cd38fc136c6523ffebd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } health_report { overall_health: HEALTHY } } }
I20260811 10:07:15.430444 25439 raft_consensus.cc:1261] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Refusing update from remote peer 23b090f894854cd38fc136c6523ffebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:15.430812 25666 consensus_queue.cc:1035] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:15.430909 25285 raft_consensus.cc:1261] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Refusing update from remote peer 23b090f894854cd38fc136c6523ffebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:15.431134 25666 consensus_queue.cc:1035] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 10:07:15.438333 25693 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 10:07:16.960253 25110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:16.960371 25110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:16.961552 25110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Sending DeleteTablet for 3 replicas of tablet 3505748360954e0e85702d290a940475
I20260811 10:07:16.961930 25415 tablet_service.cc:1514] Processing DeleteTablet for tablet 3505748360954e0e85702d290a940475 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:16 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:40192
I20260811 10:07:16.961937 25263 tablet_service.cc:1514] Processing DeleteTablet for tablet 3505748360954e0e85702d290a940475 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:16 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:39562
I20260811 10:07:16.962191 25568 tablet_service.cc:1514] Processing DeleteTablet for tablet 3505748360954e0e85702d290a940475 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:16 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:56804
I20260811 10:07:16.962193 25748 tablet_replica.cc:330] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: stopping tablet replica
I20260811 10:07:16.962325 25748 raft_consensus.cc:2229] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:16.962388 25748 raft_consensus.cc:2258] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:16.962668 25748 ts_tablet_manager.cc:1905] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:16.964530 25748 ts_tablet_manager.cc:1918] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:07:16.964610 25748 log.cc:1196] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/3505748360954e0e85702d290a940475
I20260811 10:07:16.964874 25748 ts_tablet_manager.cc:1939] T 3505748360954e0e85702d290a940475 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting consensus metadata
I20260811 10:07:16.965018 25749 tablet_replica.cc:330] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: stopping tablet replica
I20260811 10:07:16.965107 25749 raft_consensus.cc:2229] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:16.965160 25749 raft_consensus.cc:2258] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:16.965302 25750 tablet_replica.cc:330] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: stopping tablet replica
I20260811 10:07:16.965349 25095 catalog_manager.cc:4841] TS 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421): tablet 3505748360954e0e85702d290a940475 (table test-table [id=47c9ac720c3a4851881ada7aba678588]) successfully deleted
I20260811 10:07:16.965380 25750 raft_consensus.cc:2229] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:16.965453 25750 raft_consensus.cc:2258] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:16.965448 25749 ts_tablet_manager.cc:1905] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:16.965749 25750 ts_tablet_manager.cc:1905] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:16.967149 25749 ts_tablet_manager.cc:1918] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:07:16.967223 25749 log.cc:1196] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/3505748360954e0e85702d290a940475
I20260811 10:07:16.967464 25749 ts_tablet_manager.cc:1939] T 3505748360954e0e85702d290a940475 P e3819229bb2a47009f1a40d2a563512b: Deleting consensus metadata
I20260811 10:07:16.967549 25750 ts_tablet_manager.cc:1918] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:07:16.967610 25750 log.cc:1196] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/3505748360954e0e85702d290a940475
I20260811 10:07:16.967841 25750 ts_tablet_manager.cc:1939] T 3505748360954e0e85702d290a940475 P 23b090f894854cd38fc136c6523ffebd: Deleting consensus metadata
I20260811 10:07:16.968158 25096 catalog_manager.cc:4841] TS e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689): tablet 3505748360954e0e85702d290a940475 (table test-table [id=47c9ac720c3a4851881ada7aba678588]) successfully deleted
I20260811 10:07:16.968468 25094 catalog_manager.cc:4841] TS 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979): tablet 3505748360954e0e85702d290a940475 (table test-table [id=47c9ac720c3a4851881ada7aba678588]) successfully deleted
Aug 11 10:07:16 dist-test-slave-g350 krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442836, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:07:16 dist-test-slave-g350 krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442836, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:16.992573 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:16.983744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.22.177.62:45273)
0811 10:07:16.983931 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:16.983933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:16.984005 (+ 72us) server_negotiation.cc:407] Connection header received
0811 10:07:16.984112 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:16.984113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:16.984126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:16.984204 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:16.984913 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:16.985541 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:16.986436 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:16.986721 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:16.986801 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:16.989418 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:16.989421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:16.989421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:16.989431 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:16.991155 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:16.991591 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:16.991593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:16.991593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:16.991677 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:16.991914 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:16.991915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:16.991915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:16.992058 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:16.992188 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:16.992334 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 10:07:16.992443 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260811 10:07:16.994580 25110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:16.994783 25110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:16.996441 25263 tablet_service.cc:1467] Processing CreateTablet for tablet b06556e89f7546cbbad95cc1b6a6a134 (DEFAULT_TABLE table=test-table [id=1aed8ee2ab8f4f348344bb7608f2d64d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:16.996472 25568 tablet_service.cc:1467] Processing CreateTablet for tablet b06556e89f7546cbbad95cc1b6a6a134 (DEFAULT_TABLE table=test-table [id=1aed8ee2ab8f4f348344bb7608f2d64d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:16.996547 25415 tablet_service.cc:1467] Processing CreateTablet for tablet b06556e89f7546cbbad95cc1b6a6a134 (DEFAULT_TABLE table=test-table [id=1aed8ee2ab8f4f348344bb7608f2d64d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:16.996634 25263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06556e89f7546cbbad95cc1b6a6a134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:16.996645 25568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06556e89f7546cbbad95cc1b6a6a134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:16.996668 25415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06556e89f7546cbbad95cc1b6a6a134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:16.998294 25761 tablet_bootstrap.cc:492] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Bootstrap starting.
I20260811 10:07:16.998407 25760 tablet_bootstrap.cc:492] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Bootstrap starting.
I20260811 10:07:16.998732 25761 tablet_bootstrap.cc:654] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:16.998816 25760 tablet_bootstrap.cc:654] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:16.999121 25763 tablet_bootstrap.cc:492] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Bootstrap starting.
I20260811 10:07:16.999398 25761 tablet_bootstrap.cc:492] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: No bootstrap required, opened a new log
I20260811 10:07:16.999446 25761 ts_tablet_manager.cc:1397] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:16.999481 25760 tablet_bootstrap.cc:492] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: No bootstrap required, opened a new log
I20260811 10:07:16.999536 25760 ts_tablet_manager.cc:1397] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:16.999601 25761 raft_consensus.cc:348] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:16.999682 25761 raft_consensus.cc:374] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:16.999698 25761 raft_consensus.cc:729] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Initialized, Role: FOLLOWER
I20260811 10:07:16.999653 25760 raft_consensus.cc:348] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:16.999759 25760 raft_consensus.cc:374] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:16.999758 25761 consensus_queue.cc:260] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:16.999787 25760 raft_consensus.cc:729] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Initialized, Role: FOLLOWER
I20260811 10:07:16.999873 25761 ts_tablet_manager.cc:1428] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:16.999918 25760 consensus_queue.cc:260] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.000177 25760 ts_tablet_manager.cc:1428] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:17.000250 25763 tablet_bootstrap.cc:654] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:17.000905 25763 tablet_bootstrap.cc:492] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: No bootstrap required, opened a new log
I20260811 10:07:17.000974 25763 ts_tablet_manager.cc:1397] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:17.001143 25763 raft_consensus.cc:348] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.001223 25763 raft_consensus.cc:374] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:17.001248 25763 raft_consensus.cc:729] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3819229bb2a47009f1a40d2a563512b, State: Initialized, Role: FOLLOWER
I20260811 10:07:17.001305 25763 consensus_queue.cc:260] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.001479 25763 ts_tablet_manager.cc:1428] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:17.248591 25767 raft_consensus.cc:482] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:17.248724 25767 raft_consensus.cc:504] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.249327 25767 leader_election.cc:288] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421), 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:17.252946 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.249644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:37581 (local address 127.22.177.2:35421)
0811 10:07:17.249825 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.249828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.249942 (+ 114us) server_negotiation.cc:407] Connection header received
0811 10:07:17.250043 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.250044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.250056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.250147 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:17.250355 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.251059 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.252140 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.252539 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.252638 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.252650 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.252819 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.252864 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260811 10:07:17.252967 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.249541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.22.177.1:37581)
0811 10:07:17.249836 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:17.249848 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:17.249960 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:17.250151 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:17.250152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:17.250155 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:17.250273 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.250275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.251173 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.251175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.252034 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.252037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.252636 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.252637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.252728 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.252732 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:17.252799 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:07:17.252846 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":25,"thread_start_us":186,"threads_started":1}
I20260811 10:07:17.253057 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.249715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:45335 (local address 127.22.177.3:34979)
0811 10:07:17.249873 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.249875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.250015 (+ 140us) server_negotiation.cc:407] Connection header received
0811 10:07:17.250098 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.250100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.250118 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.250192 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:17.250832 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.251331 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.252271 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.252665 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.252760 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.252770 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.252938 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.252986 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260811 10:07:17.253289 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.249660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:34979 (local address 127.22.177.1:45335)
0811 10:07:17.249925 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:17.249935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:17.250026 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:17.250212 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:17.250213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:17.250216 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:17.250704 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.250706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.251433 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.251435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.252168 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.252170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.252760 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.252762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.252840 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.252843 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:17.252932 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 10:07:17.252976 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":114,"threads_started":1}
I20260811 10:07:17.253252 25439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06556e89f7546cbbad95cc1b6a6a134" candidate_uuid: "e3819229bb2a47009f1a40d2a563512b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ab3414cc14fbc942d650f9f9af5bb" is_pre_election: true
I20260811 10:07:17.253362 25439 raft_consensus.cc:2454] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3819229bb2a47009f1a40d2a563512b in term 0.
I20260811 10:07:17.253546 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06556e89f7546cbbad95cc1b6a6a134" candidate_uuid: "e3819229bb2a47009f1a40d2a563512b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b090f894854cd38fc136c6523ffebd" is_pre_election: true
I20260811 10:07:17.253548 25212 leader_election.cc:302] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947ab3414cc14fbc942d650f9f9af5bb, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:17.253686 25589 raft_consensus.cc:2454] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3819229bb2a47009f1a40d2a563512b in term 0.
I20260811 10:07:17.253748 25767 raft_consensus.cc:2781] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:17.253794 25767 raft_consensus.cc:482] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:17.253815 25767 raft_consensus.cc:3037] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:17.254694 25767 raft_consensus.cc:504] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.254858 25767 leader_election.cc:288] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [CANDIDATE]: Term 1 election: Requested vote from peers 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421), 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:17.255055 25439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06556e89f7546cbbad95cc1b6a6a134" candidate_uuid: "e3819229bb2a47009f1a40d2a563512b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ab3414cc14fbc942d650f9f9af5bb"
I20260811 10:07:17.255158 25439 raft_consensus.cc:3037] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:17.255121 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06556e89f7546cbbad95cc1b6a6a134" candidate_uuid: "e3819229bb2a47009f1a40d2a563512b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b090f894854cd38fc136c6523ffebd"
I20260811 10:07:17.255239 25589 raft_consensus.cc:3037] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:17.255918 25439 raft_consensus.cc:2454] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3819229bb2a47009f1a40d2a563512b in term 1.
I20260811 10:07:17.255929 25589 raft_consensus.cc:2454] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3819229bb2a47009f1a40d2a563512b in term 1.
I20260811 10:07:17.256131 25211 leader_election.cc:302] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b090f894854cd38fc136c6523ffebd, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:17.256304 25767 raft_consensus.cc:2781] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:17.256507 25767 raft_consensus.cc:686] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 1 LEADER]: Becoming Leader. State: Replica: e3819229bb2a47009f1a40d2a563512b, State: Running, Role: LEADER
I20260811 10:07:17.256603 25767 consensus_queue.cc:237] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.257546 25110 catalog_manager.cc:5495] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b reported cstate change: term changed from 0 to 1, leader changed from <none> to e3819229bb2a47009f1a40d2a563512b (127.22.177.1). New cstate: current_term: 1 leader_uuid: "e3819229bb2a47009f1a40d2a563512b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:17.290285 25776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.287344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.22.177.1:39689)
0811 10:07:17.287509 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.287511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.287520 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:17.287571 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.287572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.287592 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.287668 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:17.287923 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.288433 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.289343 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.289563 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.289658 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.289860 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.290013 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.290081 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.290176 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.290196 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260811 10:07:17.296216 25779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.293148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.22.177.1:59335)
0811 10:07:17.293489 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:17.293501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:17.293558 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:17.293774 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:17.293775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:17.293777 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:17.293842 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.293844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.294542 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.294543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.295157 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.295158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.295649 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.295649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.295734 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.295752 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.296007 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.296011 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:17.296076 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:07:17.296123 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":183,"threads_started":1}
I20260811 10:07:17.296168 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.293230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:59335 (local address 127.22.177.2:35421)
0811 10:07:17.293344 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.293346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.293564 (+ 218us) server_negotiation.cc:407] Connection header received
0811 10:07:17.293664 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.293665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.293686 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.293734 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:17.293920 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.294455 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.295337 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.295556 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.295620 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.295855 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.295901 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.295957 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.296081 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.296105 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 10:07:17.298920 25589 raft_consensus.cc:1261] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Refusing update from remote peer e3819229bb2a47009f1a40d2a563512b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:17.298923 25439 raft_consensus.cc:1261] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Refusing update from remote peer e3819229bb2a47009f1a40d2a563512b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:17.299229 25767 consensus_queue.cc:1035] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:17.299397 25772 consensus_queue.cc:1035] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [LEADER]: Connected to new peer: Peer: permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:17.307888 25776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.304929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:59149 (local address 127.22.177.1:39689)
0811 10:07:17.304971 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.304974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.305242 (+ 268us) server_negotiation.cc:407] Connection header received
0811 10:07:17.305346 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.305347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.305359 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.305415 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:17.305538 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.306168 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.307095 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.307404 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.307477 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.307625 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.307655 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.307693 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.307765 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.307790 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:07:17.307886 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.304887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:39689 (local address 127.22.177.2:59149)
0811 10:07:17.305138 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:17.305150 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:17.305249 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:17.305416 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:17.305417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:17.305419 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:17.305481 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.305483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.306270 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.306271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.306992 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.306995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.307475 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.307476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.307553 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.307559 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.307692 (+ 133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.307693 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:07:17.307723 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 10:07:17.307805 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":92,"threads_started":1}
I20260811 10:07:17.318749 25110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:17.318902 25110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:17.319746 25110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Sending DeleteTablet for 3 replicas of tablet b06556e89f7546cbbad95cc1b6a6a134
I20260811 10:07:17.320052 25415 tablet_service.cc:1514] Processing DeleteTablet for tablet b06556e89f7546cbbad95cc1b6a6a134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:17 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:40192
I20260811 10:07:17.320125 25263 tablet_service.cc:1514] Processing DeleteTablet for tablet b06556e89f7546cbbad95cc1b6a6a134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:17 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:39562
I20260811 10:07:17.320127 25568 tablet_service.cc:1514] Processing DeleteTablet for tablet b06556e89f7546cbbad95cc1b6a6a134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:17 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:56804
I20260811 10:07:17.320165 25748 tablet_replica.cc:330] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: stopping tablet replica
I20260811 10:07:17.320227 25750 tablet_replica.cc:330] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: stopping tablet replica
I20260811 10:07:17.320230 25748 raft_consensus.cc:2229] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:17.320284 25748 raft_consensus.cc:2258] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:17.320283 25750 raft_consensus.cc:2229] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:17.320353 25750 raft_consensus.cc:2258] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:17.320439 25749 tablet_replica.cc:330] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: stopping tablet replica
I20260811 10:07:17.320497 25749 raft_consensus.cc:2229] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:17.320598 25749 raft_consensus.cc:2258] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:17.320675 25748 ts_tablet_manager.cc:1905] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:17.320700 25110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:17.320863 25110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:17.321467 25749 ts_tablet_manager.cc:1905] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:17.322377 25568 tablet_service.cc:1467] Processing CreateTablet for tablet 62de6fb144b74a2b86145cedbd1aef15 (DEFAULT_TABLE table=test-table [id=b4a4a0166bb2413288cd2f8c92991ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:17.322503 25568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62de6fb144b74a2b86145cedbd1aef15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:17.322579 25748 ts_tablet_manager.cc:1918] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:17.322636 25748 log.cc:1196] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/b06556e89f7546cbbad95cc1b6a6a134
I20260811 10:07:17.322867 25748 ts_tablet_manager.cc:1939] T b06556e89f7546cbbad95cc1b6a6a134 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting consensus metadata
I20260811 10:07:17.323163 25749 ts_tablet_manager.cc:1918] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:17.323221 25749 log.cc:1196] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/b06556e89f7546cbbad95cc1b6a6a134
I20260811 10:07:17.323436 25749 ts_tablet_manager.cc:1939] T b06556e89f7546cbbad95cc1b6a6a134 P e3819229bb2a47009f1a40d2a563512b: Deleting consensus metadata
I20260811 10:07:17.323897 25750 ts_tablet_manager.cc:1905] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:17.324043 25760 tablet_bootstrap.cc:492] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Bootstrap starting.
I20260811 10:07:17.324378 25760 tablet_bootstrap.cc:654] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:17.324846 25263 tablet_service.cc:1467] Processing CreateTablet for tablet 62de6fb144b74a2b86145cedbd1aef15 (DEFAULT_TABLE table=test-table [id=b4a4a0166bb2413288cd2f8c92991ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:17.324962 25263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62de6fb144b74a2b86145cedbd1aef15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:17.325522 25750 ts_tablet_manager.cc:1918] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:17.325582 25750 log.cc:1196] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/b06556e89f7546cbbad95cc1b6a6a134
I20260811 10:07:17.325678 25096 catalog_manager.cc:4841] TS e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689): tablet b06556e89f7546cbbad95cc1b6a6a134 (table test-table [id=1aed8ee2ab8f4f348344bb7608f2d64d]) successfully deleted
I20260811 10:07:17.325929 25750 ts_tablet_manager.cc:1939] T b06556e89f7546cbbad95cc1b6a6a134 P 23b090f894854cd38fc136c6523ffebd: Deleting consensus metadata
I20260811 10:07:17.326064 25095 catalog_manager.cc:4841] TS 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421): tablet b06556e89f7546cbbad95cc1b6a6a134 (table test-table [id=1aed8ee2ab8f4f348344bb7608f2d64d]) successfully deleted
I20260811 10:07:17.326153 25415 tablet_service.cc:1467] Processing CreateTablet for tablet 62de6fb144b74a2b86145cedbd1aef15 (DEFAULT_TABLE table=test-table [id=b4a4a0166bb2413288cd2f8c92991ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:17.326252 25094 catalog_manager.cc:4841] TS 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979): tablet b06556e89f7546cbbad95cc1b6a6a134 (table test-table [id=1aed8ee2ab8f4f348344bb7608f2d64d]) successfully deleted
I20260811 10:07:17.326290 25415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62de6fb144b74a2b86145cedbd1aef15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:17.327335 25761 tablet_bootstrap.cc:492] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Bootstrap starting.
I20260811 10:07:17.327536 25760 tablet_bootstrap.cc:492] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: No bootstrap required, opened a new log
I20260811 10:07:17.327589 25760 ts_tablet_manager.cc:1397] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 10:07:17.327700 25761 tablet_bootstrap.cc:654] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:17.327728 25760 raft_consensus.cc:348] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.327817 25760 raft_consensus.cc:374] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:17.327863 25760 raft_consensus.cc:729] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b090f894854cd38fc136c6523ffebd, State: Initialized, Role: FOLLOWER
I20260811 10:07:17.327920 25760 consensus_queue.cc:260] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.328037 25760 ts_tablet_manager.cc:1428] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:17.328476 25761 tablet_bootstrap.cc:492] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: No bootstrap required, opened a new log
I20260811 10:07:17.328531 25761 ts_tablet_manager.cc:1397] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:17.328641 25761 raft_consensus.cc:348] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.328703 25761 raft_consensus.cc:374] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:17.328723 25761 raft_consensus.cc:729] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Initialized, Role: FOLLOWER
I20260811 10:07:17.328773 25761 consensus_queue.cc:260] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.328861 25761 ts_tablet_manager.cc:1428] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:17.329391 25763 tablet_bootstrap.cc:492] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b: Bootstrap starting.
I20260811 10:07:17.329962 25763 tablet_bootstrap.cc:654] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:17.330704 25763 tablet_bootstrap.cc:492] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b: No bootstrap required, opened a new log
I20260811 10:07:17.330758 25763 ts_tablet_manager.cc:1397] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:17.330926 25763 raft_consensus.cc:348] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.330998 25763 raft_consensus.cc:374] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:17.331022 25763 raft_consensus.cc:729] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3819229bb2a47009f1a40d2a563512b, State: Initialized, Role: FOLLOWER
I20260811 10:07:17.331070 25763 consensus_queue.cc:260] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.331178 25763 ts_tablet_manager.cc:1428] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:17.436465 25732 raft_consensus.cc:482] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:17.436594 25732 raft_consensus.cc:504] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.436762 25732 leader_election.cc:288] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:17.437033 25284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de6fb144b74a2b86145cedbd1aef15" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b" is_pre_election: true
I20260811 10:07:17.437033 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de6fb144b74a2b86145cedbd1aef15" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b090f894854cd38fc136c6523ffebd" is_pre_election: true
I20260811 10:07:17.437152 25284 raft_consensus.cc:2454] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 0.
I20260811 10:07:17.437152 25589 raft_consensus.cc:2454] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 0.
I20260811 10:07:17.437328 25366 leader_election.cc:302] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947ab3414cc14fbc942d650f9f9af5bb, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:17.437445 25732 raft_consensus.cc:2781] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:17.437500 25732 raft_consensus.cc:482] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:17.437520 25732 raft_consensus.cc:3037] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:17.438532 25732 raft_consensus.cc:504] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.438686 25732 leader_election.cc:288] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 election: Requested vote from peers e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689), 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979)
I20260811 10:07:17.438928 25284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de6fb144b74a2b86145cedbd1aef15" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3819229bb2a47009f1a40d2a563512b"
I20260811 10:07:17.438928 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de6fb144b74a2b86145cedbd1aef15" candidate_uuid: "947ab3414cc14fbc942d650f9f9af5bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b090f894854cd38fc136c6523ffebd"
I20260811 10:07:17.439021 25589 raft_consensus.cc:3037] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:17.439020 25284 raft_consensus.cc:3037] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:17.439910 25284 raft_consensus.cc:2454] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 1.
I20260811 10:07:17.439922 25589 raft_consensus.cc:2454] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947ab3414cc14fbc942d650f9f9af5bb in term 1.
I20260811 10:07:17.440119 25366 leader_election.cc:302] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947ab3414cc14fbc942d650f9f9af5bb, e3819229bb2a47009f1a40d2a563512b; no voters:
I20260811 10:07:17.440214 25732 raft_consensus.cc:2781] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:17.440258 25732 raft_consensus.cc:686] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 LEADER]: Becoming Leader. State: Replica: 947ab3414cc14fbc942d650f9f9af5bb, State: Running, Role: LEADER
I20260811 10:07:17.440308 25732 consensus_queue.cc:237] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } }
I20260811 10:07:17.441015 25110 catalog_manager.cc:5495] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2). New cstate: current_term: 1 leader_uuid: "947ab3414cc14fbc942d650f9f9af5bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947ab3414cc14fbc942d650f9f9af5bb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:17.456202 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.453682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.22.177.2:35421)
0811 10:07:17.453765 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.453767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.453777 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:17.453831 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.453832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.453850 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.453906 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:17.454029 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.454639 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.455435 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.455645 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.455731 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.455894 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.455936 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.455998 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.456084 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.456108 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:07:17.461663 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.459259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35421 (local address 127.22.177.2:56765)
0811 10:07:17.459381 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:17.459393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:17.459451 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:17.459621 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:17.459622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:17.459623 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:17.459676 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.459677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.460320 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.460321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.460870 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:17.460871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.461225 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.461226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:17.461278 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.461283 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.461467 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.461469 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:07:17.461522 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 10:07:17.461567 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 10:07:17.461663 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:17.459458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:56765 (local address 127.22.177.2:35421)
0811 10:07:17.459518 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:17.459520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:17.459526 (+ 6us) server_negotiation.cc:407] Connection header received
0811 10:07:17.459550 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:17.459551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:17.459558 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:17.459620 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:17.459748 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.460216 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.460956 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:17.461140 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:17.461223 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:17.461371 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:17.461407 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:17.461467 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:17.461526 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 10:07:17.461546 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":40}
I20260811 10:07:17.463860 25589 raft_consensus.cc:1261] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Refusing update from remote peer 947ab3414cc14fbc942d650f9f9af5bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:17.463862 25284 raft_consensus.cc:1261] T 62de6fb144b74a2b86145cedbd1aef15 P e3819229bb2a47009f1a40d2a563512b [term 1 FOLLOWER]: Refusing update from remote peer 947ab3414cc14fbc942d650f9f9af5bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:17.464092 25732 consensus_queue.cc:1035] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b090f894854cd38fc136c6523ffebd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 34979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:17.464319 25732 consensus_queue.cc:1035] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3819229bb2a47009f1a40d2a563512b" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 39689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 10:07:17.469137 25724 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 10:07:18.990813 25111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:18.990926 25111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:18.991955 25111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0eb8645a94444cf4b791fff920fe9662: Sending DeleteTablet for 3 replicas of tablet 62de6fb144b74a2b86145cedbd1aef15
I20260811 10:07:18.992306 25568 tablet_service.cc:1514] Processing DeleteTablet for tablet 62de6fb144b74a2b86145cedbd1aef15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:56804
I20260811 10:07:18.992339 25415 tablet_service.cc:1514] Processing DeleteTablet for tablet 62de6fb144b74a2b86145cedbd1aef15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:40192
I20260811 10:07:18.992344 25263 tablet_service.cc:1514] Processing DeleteTablet for tablet 62de6fb144b74a2b86145cedbd1aef15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:39562
I20260811 10:07:18.992419 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 25193
I20260811 10:07:18.994547 25815 tablet_replica.cc:330] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: stopping tablet replica
I20260811 10:07:18.994668 25815 raft_consensus.cc:2229] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:18.994796 25815 raft_consensus.cc:2258] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:18.995422 25815 ts_tablet_manager.cc:1905] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:18.995604 25814 tablet_replica.cc:330] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: stopping tablet replica
I20260811 10:07:18.995700 25814 raft_consensus.cc:2229] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:18.995846 25814 raft_consensus.cc:2258] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:18.996165 25814 ts_tablet_manager.cc:1905] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:18.997624 25815 ts_tablet_manager.cc:1918] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:07:18.997721 25815 log.cc:1196] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/62de6fb144b74a2b86145cedbd1aef15
I20260811 10:07:18.997871 25814 ts_tablet_manager.cc:1918] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:07:18.997951 25815 ts_tablet_manager.cc:1939] T 62de6fb144b74a2b86145cedbd1aef15 P 23b090f894854cd38fc136c6523ffebd: Deleting consensus metadata
I20260811 10:07:18.997967 25814 log.cc:1196] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/62de6fb144b74a2b86145cedbd1aef15
I20260811 10:07:18.998274 25814 ts_tablet_manager.cc:1939] T 62de6fb144b74a2b86145cedbd1aef15 P 947ab3414cc14fbc942d650f9f9af5bb: Deleting consensus metadata
I20260811 10:07:18.998615 25094 catalog_manager.cc:4841] TS 23b090f894854cd38fc136c6523ffebd (127.22.177.3:34979): tablet 62de6fb144b74a2b86145cedbd1aef15 (table test-table [id=b4a4a0166bb2413288cd2f8c92991ca2]) successfully deleted
I20260811 10:07:18.998718 25095 catalog_manager.cc:4841] TS 947ab3414cc14fbc942d650f9f9af5bb (127.22.177.2:35421): tablet 62de6fb144b74a2b86145cedbd1aef15 (table test-table [id=b4a4a0166bb2413288cd2f8c92991ca2]) successfully deleted
W20260811 10:07:19.002784 25096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:39689 (error 108)
I20260811 10:07:19.003075 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 25345
I20260811 10:07:19.009917 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:19.003433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:39689 (local address 127.0.0.1:42944)
0811 10:07:19.009748 (+ 6315us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:19.009846 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6268,"thread_start_us":178,"threads_started":1}
W20260811 10:07:19.010164 25096 catalog_manager.cc:4584] TS e3819229bb2a47009f1a40d2a563512b (127.22.177.1:39689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62de6fb144b74a2b86145cedbd1aef15: Network error: Client connection negotiation failed: client connection to 127.22.177.1:39689: connect: Connection refused (error 111)
I20260811 10:07:19.016350 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 25496
I20260811 10:07:19.027204 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 25074
2026-08-11T10:07:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6150 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:19 dist-test-slave-g350 krb5kdc[25821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:19 dist-test-slave-g350 krb5kdc[25821](info): set up 2 sockets
Aug 11 10:07:19 dist-test-slave-g350 krb5kdc[25821](info): commencing operation
krb5kdc: starting...
W20260811 10:07:20.203645 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:20.352674 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42763
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:42763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:20.438925 25835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:20.439203 25835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:20.439248 25835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:20.440481 25835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:20.440565 25835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:20.440589 25835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:20.440611 25835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:20.440640 25835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:20.442435 25835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:42763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:20.442762 25835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:20.443022 25835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:20.449478 25845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:20.449486 25843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:20.449584 25835 server_base.cc:1034] running on GCE node
W20260811 10:07:20.449487 25842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:20.450107 25835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:20.450466 25835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:20.451651 25835 hybrid_clock.cc:648] HybridClock initialized: now 1786442840451633 us; error 43 us; skew 500 ppm
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:20.454748 25835 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:20.455345 25835 webserver.cc:460] Webserver started at http://127.22.177.62:42953/ using document root <none> and password file <none>
I20260811 10:07:20.455611 25835 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:20.455684 25835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:20.455850 25835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:20.457036 25835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "b232587d578f468a980fbb8647eebe0c"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "97d2dbb9a4316b18420b511db5315726"
server_key_iv: "ca0c1641155e2b989a39cd0f942dc48a"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.457429 25835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "b232587d578f468a980fbb8647eebe0c"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "97d2dbb9a4316b18420b511db5315726"
server_key_iv: "ca0c1641155e2b989a39cd0f942dc48a"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.459474 25835 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 10:07:20.460457 25853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.460707 25835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:20.460835 25835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "b232587d578f468a980fbb8647eebe0c"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "97d2dbb9a4316b18420b511db5315726"
server_key_iv: "ca0c1641155e2b989a39cd0f942dc48a"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.460919 25835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:20.476148 25835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:20.479045 25835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:20.479343 25835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:20.484272 25835 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:42763
I20260811 10:07:20.484326 25913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:42763 every 8 connection(s)
I20260811 10:07:20.484759 25835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:20.485692 25914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:20.488157 25914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c: Bootstrap starting.
I20260811 10:07:20.488791 25914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:20.489099 25914 log.cc:824] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:20.489533 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 25835
I20260811 10:07:20.489661 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:20.489846 23236 external_mini_cluster.cc:1428] Setting key bdf8f1938e1b413268217b379f1b7d0c
I20260811 10:07:20.490017 25914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c: No bootstrap required, opened a new log
I20260811 10:07:20.491900 25914 raft_consensus.cc:348] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b232587d578f468a980fbb8647eebe0c" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42763 } }
I20260811 10:07:20.492064 25914 raft_consensus.cc:374] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:20.492110 25914 raft_consensus.cc:729] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b232587d578f468a980fbb8647eebe0c, State: Initialized, Role: FOLLOWER
I20260811 10:07:20.492256 25914 consensus_queue.cc:260] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b232587d578f468a980fbb8647eebe0c" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42763 } }
I20260811 10:07:20.492352 25914 raft_consensus.cc:388] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:20.492403 25914 raft_consensus.cc:482] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:20.492453 25914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:20.493309 25914 raft_consensus.cc:504] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b232587d578f468a980fbb8647eebe0c" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42763 } }
I20260811 10:07:20.493435 25914 leader_election.cc:302] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b232587d578f468a980fbb8647eebe0c; no voters:
I20260811 10:07:20.493635 25914 leader_election.cc:288] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:20.493681 25919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:20.493805 25919 raft_consensus.cc:686] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [term 1 LEADER]: Becoming Leader. State: Replica: b232587d578f468a980fbb8647eebe0c, State: Running, Role: LEADER
I20260811 10:07:20.493892 25919 consensus_queue.cc:237] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b232587d578f468a980fbb8647eebe0c" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42763 } }
I20260811 10:07:20.493932 25914 sys_catalog.cc:564] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:20.494256 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b232587d578f468a980fbb8647eebe0c. Latest consensus state: current_term: 1 leader_uuid: "b232587d578f468a980fbb8647eebe0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b232587d578f468a980fbb8647eebe0c" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42763 } } }
I20260811 10:07:20.494268 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b232587d578f468a980fbb8647eebe0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b232587d578f468a980fbb8647eebe0c" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42763 } } }
I20260811 10:07:20.494522 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:20.494346 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c [sys.catalog]: This master's current role is: LEADER
W20260811 10:07:20.496039 25936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:20.496176 25936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:20.496275 25928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:20.496474 25928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:20.498392 25928 catalog_manager.cc:1347] Generated new cluster ID: 82bfaef1313d4855a7179fc69413418d
I20260811 10:07:20.498468 25928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:20.504179 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:20.490428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.22.177.62:42763)
0811 10:07:20.490617 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:20.490626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:20.490678 (+ 52us) server_negotiation.cc:407] Connection header received
0811 10:07:20.491905 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:20.491914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:20.491984 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:20.492184 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:20.493023 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.494570 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.495952 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.496677 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.496828 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:20.499887 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:20.499893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:20.499894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:20.499907 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:20.502185 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:20.502679 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:20.502683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:20.502684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:20.502870 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:20.503110 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:20.503111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:20.503111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:20.503486 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:20.503658 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:20.503773 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 10:07:20.503946 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260811 10:07:20.517051 25928 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:20.517902 25928 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:20.521435 25928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b232587d578f468a980fbb8647eebe0c: Generated new TSK 0
I20260811 10:07:20.521728 25928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:20.557909 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42763
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:20.646199 25942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:20.646409 25942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:20.646435 25942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:20.647650 25942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:20.647696 25942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:20.647724 25942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:20.649240 25942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:20.649497 25942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:20.649793 25942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:20.656283 25949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:20.656291 25948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:20.656342 25942 server_base.cc:1034] running on GCE node
W20260811 10:07:20.656283 25952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:20.656847 25942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:20.657132 25942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:20.658316 25942 hybrid_clock.cc:648] HybridClock initialized: now 1786442840658295 us; error 45 us; skew 500 ppm
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:20.661145 25942 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:20.661727 25942 webserver.cc:460] Webserver started at http://127.22.177.1:40559/ using document root <none> and password file <none>
I20260811 10:07:20.661958 25942 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:20.662003 25942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:20.662099 25942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:20.663159 25942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "45d18c81ba8c41f8acc4e2dba9423e97"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "b9b23a26390995599710dbc30f216f3f"
server_key_iv: "1bf6909646545a078626a1aeb2bae431"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.663502 25942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "45d18c81ba8c41f8acc4e2dba9423e97"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "b9b23a26390995599710dbc30f216f3f"
server_key_iv: "1bf6909646545a078626a1aeb2bae431"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.665112 25942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:20.666059 25959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.666241 25942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:20.666368 25942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "45d18c81ba8c41f8acc4e2dba9423e97"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "b9b23a26390995599710dbc30f216f3f"
server_key_iv: "1bf6909646545a078626a1aeb2bae431"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.666433 25942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:20.679243 25942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:20.681456 25942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:20.681581 25942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:20.681897 25942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:20.682185 25942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:20.682216 25942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.682240 25942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:20.682252 25942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.688179 25942 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:37641
I20260811 10:07:20.688349 26085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:37641 every 8 connection(s)
I20260811 10:07:20.688623 25942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:20.694262 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 25942
I20260811 10:07:20.694358 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:20.694587 23236 external_mini_cluster.cc:1428] Setting key 9398100c1323bf73bd3af1e9250b4515
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:20.699455 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:20.689480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:36443 (local address 127.22.177.62:42763)
0811 10:07:20.689557 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:20.689559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:20.689806 (+ 247us) server_negotiation.cc:407] Connection header received
0811 10:07:20.690306 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:20.690307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:20.690322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:20.690420 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:20.691471 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.691969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.692981 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.693187 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.693244 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:20.696102 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:20.696104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:20.696104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:20.696110 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:20.697950 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:20.698385 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:20.698386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:20.698387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:20.698462 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:20.698713 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:20.698714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:20.698715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:20.698869 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:20.698941 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:20.699212 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 10:07:20.699313 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:07:20.699483 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:20.689432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42763 (local address 127.22.177.1:36443)
0811 10:07:20.689661 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:20.689710 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:20.690219 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:20.690502 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:20.690506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:20.691102 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:20.691377 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:20.691383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.692049 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.692050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:20.692890 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:20.692893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.693315 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.693316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:20.693417 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:20.693981 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:20.693990 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:20.696004 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:20.698041 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:20.698049 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:20.698051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:20.698281 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:20.698537 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:20.698538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:20.698538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:20.698640 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:20.698944 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:20.698950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:20.699076 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 10:07:20.699166 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 10:07:20.699292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":133,"threads_started":1}
I20260811 10:07:20.700071 26086 heartbeater.cc:347] Connected to a master server at 127.22.177.62:42763
I20260811 10:07:20.700209 26086 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:20.700461 26086 heartbeater.cc:511] Master 127.22.177.62:42763 requested a full tablet report, sending...
I20260811 10:07:20.701159 25870 ts_manager.cc:194] Registered new tserver with Master: 45d18c81ba8c41f8acc4e2dba9423e97 (127.22.177.1:37641)
I20260811 10:07:20.702199 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:36443
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:20.725812 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42763
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:20.818576 26092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:20.818815 26092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:20.818849 26092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:20.820011 26092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:20.820058 26092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:20.820083 26092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:20.821538 26092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:20.821854 26092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:20.822059 26092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:20.827981 26100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:20.827981 26097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:20.827983 26098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:20.828205 26092 server_base.cc:1034] running on GCE node
I20260811 10:07:20.828529 26092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:20.828771 26092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:20.829922 26092 hybrid_clock.cc:648] HybridClock initialized: now 1786442840829884 us; error 50 us; skew 500 ppm
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:20.832939 26092 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:20.833433 26092 webserver.cc:460] Webserver started at http://127.22.177.2:33853/ using document root <none> and password file <none>
I20260811 10:07:20.833765 26092 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:20.833837 26092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:20.833997 26092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:20.835100 26092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "13446c1023d54f2180963f722455a027"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "9ac9f8f1c128aaed35bf05bdd8da579f"
server_key_iv: "24e9b00469a676093597676cf7d37604"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.835532 26092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "13446c1023d54f2180963f722455a027"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "9ac9f8f1c128aaed35bf05bdd8da579f"
server_key_iv: "24e9b00469a676093597676cf7d37604"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.837003 26092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:20.838083 26109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.838395 26092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:20.838517 26092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "13446c1023d54f2180963f722455a027"
format_stamp: "Formatted at 2026-08-11 10:07:20 on dist-test-slave-g350"
server_key: "9ac9f8f1c128aaed35bf05bdd8da579f"
server_key_iv: "24e9b00469a676093597676cf7d37604"
server_key_version: "encryptionkey@0"
I20260811 10:07:20.838613 26092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:20.858799 26092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:20.861658 26092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:20.861865 26092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:20.862182 26092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:20.862579 26092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:20.862644 26092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.862682 26092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:20.862706 26092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:20.870105 26092 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:38695
I20260811 10:07:20.870239 26234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:38695 every 8 connection(s)
I20260811 10:07:20.870581 26092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:20.872669 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26092
I20260811 10:07:20.872834 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:20.873059 23236 external_mini_cluster.cc:1428] Setting key b0e3d2dbeb0280c71f952f97f2f07db5
Aug 11 10:07:20 dist-test-slave-g350 krb5kdc[25821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442840, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:20.883050 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:20.871472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:54781 (local address 127.22.177.62:42763)
0811 10:07:20.871553 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:20.871555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:20.871795 (+ 240us) server_negotiation.cc:407] Connection header received
0811 10:07:20.872797 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:20.872799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:20.872815 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:20.872902 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:20.873899 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.874717 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.875978 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.876170 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.876257 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:20.879484 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:20.879486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:20.879486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:20.879495 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:20.881189 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:20.881786 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:20.881787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:20.881788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:20.881853 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:20.882286 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:20.882287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:20.882287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:20.882415 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:20.882475 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:20.882882 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 10:07:20.882969 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:20.883323 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:20.871441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42763 (local address 127.22.177.2:54781)
0811 10:07:20.871624 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:20.871671 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:20.872665 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:20.872999 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:20.873003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:20.873459 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:20.873771 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:20.873777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.874857 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.874858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:20.875857 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:20.875876 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:20.876349 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:20.876350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:20.876512 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:20.877303 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:20.877313 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:20.879337 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:20.881296 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:20.881307 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:20.881310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:20.881695 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:20.881939 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:20.881940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:20.881940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:20.882098 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:20.882498 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:20.882506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:20.882706 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 10:07:20.882843 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 10:07:20.883033 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260811 10:07:20.883846 26235 heartbeater.cc:347] Connected to a master server at 127.22.177.62:42763
I20260811 10:07:20.883976 26235 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:20.884188 26235 heartbeater.cc:511] Master 127.22.177.62:42763 requested a full tablet report, sending...
I20260811 10:07:20.884635 25870 ts_manager.cc:194] Registered new tserver with Master: 13446c1023d54f2180963f722455a027 (127.22.177.2:38695)
I20260811 10:07:20.885070 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:54781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:20.906077 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42763
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:20.992815 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:20.993062 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:20.993104 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:20.994457 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:20.994520 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:20.994566 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:20.996372 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:20.996647 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:20.996888 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:21.002697 26246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:21.002672 26245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:21.002781 26248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:21.003027 26240 server_base.cc:1034] running on GCE node
I20260811 10:07:21.003358 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:21.003613 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:21.004767 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786442841004757 us; error 34 us; skew 500 ppm
Aug 11 10:07:21 dist-test-slave-g350 krb5kdc[25821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442841, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:21.007934 26240 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:21.008471 26240 webserver.cc:460] Webserver started at http://127.22.177.3:33049/ using document root <none> and password file <none>
I20260811 10:07:21.008708 26240 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:21.008777 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:21.008879 26240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:21.010079 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "69552feb570f4a5e9b0085c721182bac"
format_stamp: "Formatted at 2026-08-11 10:07:21 on dist-test-slave-g350"
server_key: "0cd5a44772a29a2195cfaeb069f6cf7c"
server_key_iv: "61e12443d9ed8dc89751b1e68c3c2ed5"
server_key_version: "encryptionkey@0"
I20260811 10:07:21.010396 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "69552feb570f4a5e9b0085c721182bac"
format_stamp: "Formatted at 2026-08-11 10:07:21 on dist-test-slave-g350"
server_key: "0cd5a44772a29a2195cfaeb069f6cf7c"
server_key_iv: "61e12443d9ed8dc89751b1e68c3c2ed5"
server_key_version: "encryptionkey@0"
I20260811 10:07:21.011776 26240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:21.012653 26255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:21.012900 26240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:21.013006 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "69552feb570f4a5e9b0085c721182bac"
format_stamp: "Formatted at 2026-08-11 10:07:21 on dist-test-slave-g350"
server_key: "0cd5a44772a29a2195cfaeb069f6cf7c"
server_key_iv: "61e12443d9ed8dc89751b1e68c3c2ed5"
server_key_version: "encryptionkey@0"
I20260811 10:07:21.013114 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:21.032161 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:21.034765 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:21.034977 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:21.035249 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:21.035638 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:21.035694 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:21.035739 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:21.035763 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:21.042186 26240 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:35741
I20260811 10:07:21.042253 26378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:35741 every 8 connection(s)
I20260811 10:07:21.042613 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:21.042982 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26240
I20260811 10:07:21.043097 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:21.043311 23236 external_mini_cluster.cc:1428] Setting key 26ff8e6d5888b00bbfe5849a43dce556
Aug 11 10:07:21 dist-test-slave-g350 krb5kdc[25821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442841, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:21.053833 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:21.043883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:46191 (local address 127.22.177.62:42763)
0811 10:07:21.043934 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:21.043937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:21.043947 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:21.044603 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:21.044604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:21.044614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:21.044662 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:21.045690 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:21.046220 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:21.047269 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:21.047490 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:21.047552 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:21.050254 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:21.050255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:21.050255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:21.050263 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:21.051948 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:21.052535 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:21.052539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:21.052539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:21.052625 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:21.052979 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:21.052981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:21.052981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:21.053138 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:21.053214 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:21.053578 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 10:07:21.053759 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:07:21.054018 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:21.043542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42763 (local address 127.22.177.3:46191)
0811 10:07:21.043786 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:21.043838 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:21.044511 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:21.044723 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:21.044726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:21.045354 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:21.045557 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:21.045561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:21.046318 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:21.046319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:21.047190 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:21.047193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:21.047566 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:21.047567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:21.047668 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:21.048296 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:21.048305 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:21.050123 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:21.052059 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:21.052068 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:21.052070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:21.052318 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:21.052744 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:21.052745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:21.052746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:21.052877 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:21.053278 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:21.053284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:21.053424 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 10:07:21.053522 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 10:07:21.053769 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":125,"threads_started":1}
I20260811 10:07:21.054602 26379 heartbeater.cc:347] Connected to a master server at 127.22.177.62:42763
I20260811 10:07:21.054734 26379 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:21.054953 26379 heartbeater.cc:511] Master 127.22.177.62:42763 requested a full tablet report, sending...
I20260811 10:07:21.055351 25869 ts_manager.cc:194] Registered new tserver with Master: 69552feb570f4a5e9b0085c721182bac (127.22.177.3:35741)
I20260811 10:07:21.055771 25869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:46191
I20260811 10:07:21.066824 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 10:07:21.081844 25918 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:21.080597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.22.177.62:42763)
0811 10:07:21.080679 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:21.080681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:21.080796 (+ 115us) server_negotiation.cc:407] Connection header received
0811 10:07:21.080851 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:21.080852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:21.080863 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:21.080936 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:21.081727 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60804: BlockingRecv error: recv got EOF from 127.0.0.1:60804 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 10:07:21.082835 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 25942
I20260811 10:07:21.090047 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26092
I20260811 10:07:21.100348 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26240
I20260811 10:07:21.109755 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 25835
2026-08-11T10:07:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2081 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:21 dist-test-slave-g350 krb5kdc[26391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:21 dist-test-slave-g350 krb5kdc[26391](info): set up 2 sockets
Aug 11 10:07:21 dist-test-slave-g350 krb5kdc[26391](info): commencing operation
krb5kdc: starting...
W20260811 10:07:22.263690 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:22 dist-test-slave-g350 krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:22.406729 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35513
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:35513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:22.495030 26409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:22.495263 26409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:22.495296 26409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:22.496557 26409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:22.496605 26409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:22.496620 26409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:22.496634 26409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:22.496645 26409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:22.498246 26409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:35513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:22.498528 26409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:22.498796 26409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:22.504237 26414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:22.504287 26417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:22.504628 26409 server_base.cc:1034] running on GCE node
W20260811 10:07:22.504244 26415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:22.504938 26409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:22.505206 26409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:22.506393 26409 hybrid_clock.cc:648] HybridClock initialized: now 1786442842506377 us; error 28 us; skew 500 ppm
Aug 11 10:07:22 dist-test-slave-g350 krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442842, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:22.509219 26409 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:22.509878 26409 webserver.cc:460] Webserver started at http://127.22.177.62:40015/ using document root <none> and password file <none>
I20260811 10:07:22.510126 26409 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:22.510198 26409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:22.510358 26409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:22.511485 26409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "0c38e9a2fea24a7aa008b23e329cba8f"
format_stamp: "Formatted at 2026-08-11 10:07:22 on dist-test-slave-g350"
server_key: "0da82f33f17574a803ed3068c90d7ff5"
server_key_iv: "a35242f5b263b9dfcc740acd482afb95"
server_key_version: "encryptionkey@0"
I20260811 10:07:22.511871 26409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "0c38e9a2fea24a7aa008b23e329cba8f"
format_stamp: "Formatted at 2026-08-11 10:07:22 on dist-test-slave-g350"
server_key: "0da82f33f17574a803ed3068c90d7ff5"
server_key_iv: "a35242f5b263b9dfcc740acd482afb95"
server_key_version: "encryptionkey@0"
I20260811 10:07:22.513382 26409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:22.514338 26424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:22.514576 26409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:22.514691 26409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "0c38e9a2fea24a7aa008b23e329cba8f"
format_stamp: "Formatted at 2026-08-11 10:07:22 on dist-test-slave-g350"
server_key: "0da82f33f17574a803ed3068c90d7ff5"
server_key_iv: "a35242f5b263b9dfcc740acd482afb95"
server_key_version: "encryptionkey@0"
I20260811 10:07:22.514778 26409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:22.537758 26409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:22.540470 26409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:22.540679 26409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:22.545866 26409 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:35513
I20260811 10:07:22.545938 26479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:35513 every 8 connection(s)
I20260811 10:07:22.546653 26409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:22.547609 26480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:22.550158 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f: Bootstrap starting.
I20260811 10:07:22.550832 26480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:22.551167 26480 log.cc:824] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:22.552006 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f: No bootstrap required, opened a new log
I20260811 10:07:22.553543 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26409
I20260811 10:07:22.553680 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:22.553889 23236 external_mini_cluster.cc:1428] Setting key 27820519db5f5e8229c71a42e32755df
I20260811 10:07:22.554479 26480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35513 } }
I20260811 10:07:22.554700 26480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:22.554769 26480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c38e9a2fea24a7aa008b23e329cba8f, State: Initialized, Role: FOLLOWER
I20260811 10:07:22.554914 26480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35513 } }
I20260811 10:07:22.555014 26480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:22.555052 26480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:22.555109 26480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:22.555759 26480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35513 } }
I20260811 10:07:22.555905 26480 leader_election.cc:302] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c38e9a2fea24a7aa008b23e329cba8f; no voters:
I20260811 10:07:22.556104 26480 leader_election.cc:288] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:22.556172 26486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:22.556409 26480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:22.556418 26486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [term 1 LEADER]: Becoming Leader. State: Replica: 0c38e9a2fea24a7aa008b23e329cba8f, State: Running, Role: LEADER
I20260811 10:07:22.556569 26486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35513 } }
I20260811 10:07:22.556993 26487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35513 } } }
I20260811 10:07:22.557087 26487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:22.557262 26488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c38e9a2fea24a7aa008b23e329cba8f. Latest consensus state: current_term: 1 leader_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c38e9a2fea24a7aa008b23e329cba8f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35513 } } }
I20260811 10:07:22.557336 26488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:22.557456 26495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:22.557976 26495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:22.560029 26495 catalog_manager.cc:1347] Generated new cluster ID: bbe5b4ca937e448798998b671cb10c46
I20260811 10:07:22.560091 26495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:07:22 dist-test-slave-g350 krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:22.567828 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:22.554574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.22.177.62:35513)
0811 10:07:22.555279 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:22.555288 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:22.555325 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:07:22.555435 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:22.555441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:22.555486 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:22.555640 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:22.556626 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:22.558145 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:22.559108 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:22.559790 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:22.559894 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:22.562947 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:22.562951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:22.562952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:22.562966 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:22.565817 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:22.566299 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:22.566303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:22.566303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:22.566440 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:22.566677 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:22.566678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:22.566678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:22.567007 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:22.567163 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:22.567311 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 10:07:22.567566 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":578,"threads_started":1}
I20260811 10:07:22.569099 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:07:22.572643 26495 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:22.573288 26495 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:22.577111 26495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c38e9a2fea24a7aa008b23e329cba8f: Generated new TSK 0
I20260811 10:07:22.577348 26495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:07:22.591552 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:22.586912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59810 (local address 127.22.177.62:35513)
0811 10:07:22.587000 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:22.587003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:22.587111 (+ 108us) server_negotiation.cc:407] Connection header received
0811 10:07:22.587212 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:22.587213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:22.587224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:22.587281 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:22.587857 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:22.588418 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:22.589735 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:22.589979 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:22.590042 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:22.590973 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:22.590974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:22.590974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:22.591266 (+ 292us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:22.591284 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:22.591292 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:22.591350 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:22.591452 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 10:07:22.591485 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 10:07:22.594946 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26409
2026-08-11T10:07:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1482 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:22 dist-test-slave-g350 krb5kdc[26515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:22 dist-test-slave-g350 krb5kdc[26515](info): set up 2 sockets
Aug 11 10:07:22 dist-test-slave-g350 krb5kdc[26515](info): commencing operation
krb5kdc: starting...
W20260811 10:07:23.712594 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:23 dist-test-slave-g350 krb5kdc[26515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:23.854089 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:46479
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:46479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:23.940003 26529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:23.940337 26529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:23.940387 26529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:23.942416 26529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:23.942497 26529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:23.942525 26529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:23.942553 26529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:23.942584 26529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:23.945672 26529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:46479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:23.946038 26529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:23.946352 26529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:23.953961 26538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:23.953966 26536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:23.953961 26535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:23.954147 26529 server_base.cc:1034] running on GCE node
I20260811 10:07:23.954825 26529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:23.955245 26529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:23.956415 26529 hybrid_clock.cc:648] HybridClock initialized: now 1786442843956376 us; error 62 us; skew 500 ppm
Aug 11 10:07:23 dist-test-slave-g350 krb5kdc[26515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442843, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:23.960382 26529 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:23.961573 26529 webserver.cc:460] Webserver started at http://127.22.177.62:39139/ using document root <none> and password file <none>
I20260811 10:07:23.961899 26529 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:23.961982 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:23.962136 26529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:23.963766 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "9568df3a9a6847a98b88182c03716938"
format_stamp: "Formatted at 2026-08-11 10:07:23 on dist-test-slave-g350"
server_key: "9f59fca15c204eaa114c098c8a74fcc2"
server_key_iv: "dce2eefb2cafa366a251f5b4e3d608a8"
server_key_version: "encryptionkey@0"
I20260811 10:07:23.964227 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "9568df3a9a6847a98b88182c03716938"
format_stamp: "Formatted at 2026-08-11 10:07:23 on dist-test-slave-g350"
server_key: "9f59fca15c204eaa114c098c8a74fcc2"
server_key_iv: "dce2eefb2cafa366a251f5b4e3d608a8"
server_key_version: "encryptionkey@0"
I20260811 10:07:23.966261 26529 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 10:07:23.967490 26546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:23.967947 26529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:23.968070 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "9568df3a9a6847a98b88182c03716938"
format_stamp: "Formatted at 2026-08-11 10:07:23 on dist-test-slave-g350"
server_key: "9f59fca15c204eaa114c098c8a74fcc2"
server_key_iv: "dce2eefb2cafa366a251f5b4e3d608a8"
server_key_version: "encryptionkey@0"
I20260811 10:07:23.968194 26529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:23.987493 26529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:23.990139 26529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:23.990274 26529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:23.994621 26529 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:46479
I20260811 10:07:23.994769 26603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:46479 every 8 connection(s)
I20260811 10:07:23.995079 26529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:23.996351 26604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:23.999193 26604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938: Bootstrap starting.
I20260811 10:07:23.999908 26604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:24.000268 26604 log.cc:824] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:24.000851 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26529
I20260811 10:07:24.000989 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:24.001235 26604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938: No bootstrap required, opened a new log
I20260811 10:07:24.001329 23236 external_mini_cluster.cc:1428] Setting key b573d68b760a64803b6623a6a05ed6e8
I20260811 10:07:24.003249 26604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9568df3a9a6847a98b88182c03716938" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46479 } }
I20260811 10:07:24.003454 26604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:24.003496 26604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9568df3a9a6847a98b88182c03716938, State: Initialized, Role: FOLLOWER
I20260811 10:07:24.003664 26604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9568df3a9a6847a98b88182c03716938" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46479 } }
I20260811 10:07:24.003762 26604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:24.003810 26604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:24.003870 26604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:24.004851 26604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9568df3a9a6847a98b88182c03716938" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46479 } }
I20260811 10:07:24.004981 26604 leader_election.cc:302] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9568df3a9a6847a98b88182c03716938; no voters:
I20260811 10:07:24.005188 26604 leader_election.cc:288] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:24.005242 26609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:24.005402 26609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [term 1 LEADER]: Becoming Leader. State: Replica: 9568df3a9a6847a98b88182c03716938, State: Running, Role: LEADER
I20260811 10:07:24.005502 26604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:24.005491 26609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9568df3a9a6847a98b88182c03716938" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46479 } }
I20260811 10:07:24.006407 26611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9568df3a9a6847a98b88182c03716938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9568df3a9a6847a98b88182c03716938" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46479 } } }
I20260811 10:07:24.006428 26610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9568df3a9a6847a98b88182c03716938. Latest consensus state: current_term: 1 leader_uuid: "9568df3a9a6847a98b88182c03716938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9568df3a9a6847a98b88182c03716938" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 46479 } } }
I20260811 10:07:24.006547 26611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:24.006556 26610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:24.006862 26619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:24.007469 26619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:24.009176 26619 catalog_manager.cc:1347] Generated new cluster ID: 1a9201b49fd24198ad92f64869aa5406
I20260811 10:07:24.009243 26619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:24.017117 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.002079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.22.177.62:46479)
0811 10:07:24.002320 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:24.002329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:24.002362 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:07:24.003305 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:24.003314 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:24.003383 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:24.003588 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:24.004445 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.005884 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.007229 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.008126 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.008347 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.011826 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:24.011831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:24.011832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:24.011848 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:24.014317 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.014785 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.014789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.014790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.014956 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.015194 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.015195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.015195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.015626 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:24.016585 (+ 959us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:24.016674 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 10:07:24.016861 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260811 10:07:24.018155 26619 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:24.018684 26619 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:24.025305 26619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9568df3a9a6847a98b88182c03716938: Generated new TSK 0
I20260811 10:07:24.025528 26619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:24.066138 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46479
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:24.155776 26634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:24.156019 26634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:24.156045 26634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:24.157192 26634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:24.157229 26634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:24.157338 26634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:24.158939 26634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:24.159148 26634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:24.159345 26634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:24.164655 26640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:24.164695 26639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:24.164971 26634 server_base.cc:1034] running on GCE node
W20260811 10:07:24.164692 26642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:24.165318 26634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:24.165529 26634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:24.166707 26634 hybrid_clock.cc:648] HybridClock initialized: now 1786442844166678 us; error 44 us; skew 500 ppm
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:24.174206 26634 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:24.174906 26634 webserver.cc:460] Webserver started at http://127.22.177.1:38975/ using document root <none> and password file <none>
I20260811 10:07:24.175197 26634 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:24.175287 26634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:24.175469 26634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:24.176738 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "a823c43b98d34a98978f1112da1a73af"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "2070e1cc48e3190aebdaa83932b1b8eb"
server_key_iv: "6ba30584fb15d94e5f8f19164a298c9d"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.177138 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "a823c43b98d34a98978f1112da1a73af"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "2070e1cc48e3190aebdaa83932b1b8eb"
server_key_iv: "6ba30584fb15d94e5f8f19164a298c9d"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.178910 26634 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 10:07:24.180003 26649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.180194 26634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:24.180299 26634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "a823c43b98d34a98978f1112da1a73af"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "2070e1cc48e3190aebdaa83932b1b8eb"
server_key_iv: "6ba30584fb15d94e5f8f19164a298c9d"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.180364 26634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:24.192947 26634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:24.195259 26634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:24.195384 26634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:24.195578 26634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:24.195900 26634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:24.195947 26634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.195973 26634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:24.195986 26634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.202379 26634 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:36057
I20260811 10:07:24.202489 26771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:36057 every 8 connection(s)
I20260811 10:07:24.202824 26634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:24.212716 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26634
I20260811 10:07:24.212874 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:24.213091 23236 external_mini_cluster.cc:1428] Setting key 0a5acbe662c93320c1f08213189b92c1
I20260811 10:07:24.214254 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.203597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:51435 (local address 127.22.177.62:46479)
0811 10:07:24.203670 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:24.203672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:24.203916 (+ 244us) server_negotiation.cc:407] Connection header received
0811 10:07:24.204484 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:24.204485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:24.204499 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:24.204552 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:24.205649 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.206185 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.207435 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.207667 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.207756 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.210448 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:24.210450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:24.210450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:24.210459 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:24.212552 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.213137 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.213139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.213139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.213216 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.213511 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.213512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.213513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.213674 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:24.213742 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:24.214094 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 10:07:24.214181 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:07:24.214615 26775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.203560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:46479 (local address 127.22.177.1:51435)
0811 10:07:24.203789 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:24.203827 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:24.204356 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:24.204628 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:24.204632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:24.205224 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:24.205488 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:24.205493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.206288 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.206289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:24.207332 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:24.207335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.207765 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.207766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:24.207929 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.208542 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:24.208550 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:24.210331 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:24.212656 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:24.212668 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:24.212671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:24.213050 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:24.213289 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:24.213290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:24.213290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:24.213447 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:24.213733 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:24.213740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:24.213930 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 10:07:24.214077 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 10:07:24.214320 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260811 10:07:24.215373 26773 heartbeater.cc:347] Connected to a master server at 127.22.177.62:46479
I20260811 10:07:24.215509 26773 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:24.215718 26773 heartbeater.cc:511] Master 127.22.177.62:46479 requested a full tablet report, sending...
I20260811 10:07:24.216362 26564 ts_manager.cc:194] Registered new tserver with Master: a823c43b98d34a98978f1112da1a73af (127.22.177.1:36057)
I20260811 10:07:24.216953 26564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:51435
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:24.246269 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46479
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:24.338064 26779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:24.338289 26779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:24.338316 26779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:24.339499 26779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:24.339540 26779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:24.339568 26779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:24.341120 26779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:24.341356 26779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:24.341552 26779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:24.347304 26786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:24.347342 26789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:24.347386 26787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:24.347550 26779 server_base.cc:1034] running on GCE node
I20260811 10:07:24.347820 26779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:24.348110 26779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:24.349291 26779 hybrid_clock.cc:648] HybridClock initialized: now 1786442844349259 us; error 63 us; skew 500 ppm
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:24.352007 26779 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:24.352589 26779 webserver.cc:460] Webserver started at http://127.22.177.2:35409/ using document root <none> and password file <none>
I20260811 10:07:24.352869 26779 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:24.352945 26779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:24.353109 26779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:24.354292 26779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "e8cf10d00e9e47439f8e1e76959f9e06"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "42ea1756cad00bc7a606bbcde2f76665"
server_key_iv: "a6f0ee891e3ef9f9685acf3b06180a0b"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.354678 26779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "e8cf10d00e9e47439f8e1e76959f9e06"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "42ea1756cad00bc7a606bbcde2f76665"
server_key_iv: "a6f0ee891e3ef9f9685acf3b06180a0b"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.356149 26779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:07:24.357082 26797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.357292 26779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:24.357451 26779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "e8cf10d00e9e47439f8e1e76959f9e06"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "42ea1756cad00bc7a606bbcde2f76665"
server_key_iv: "a6f0ee891e3ef9f9685acf3b06180a0b"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.357534 26779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:24.369987 26779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:24.372437 26779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:24.372640 26779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:24.372913 26779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:24.373260 26779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:24.373317 26779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.373373 26779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:24.373397 26779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.380159 26779 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:39843
I20260811 10:07:24.380203 26928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:39843 every 8 connection(s)
I20260811 10:07:24.380569 26779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:24.382797 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26779
I20260811 10:07:24.382921 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:24.383128 23236 external_mini_cluster.cc:1428] Setting key 68c03d7ce0fa21ed8c2c91e7c8dd4c4f
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:24.393819 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.381427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:34381 (local address 127.22.177.62:46479)
0811 10:07:24.381539 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:24.381543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:24.381788 (+ 245us) server_negotiation.cc:407] Connection header received
0811 10:07:24.382394 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:24.382395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:24.382410 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:24.382476 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:24.383957 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.384536 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.386452 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.386775 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.386858 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.390213 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:24.390215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:24.390215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:24.390223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:24.391835 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.392470 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.392471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.392471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.392525 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.392921 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.392922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.392923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.393082 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:24.393228 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:24.393573 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 10:07:24.393747 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 10:07:24.394121 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.381338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:46479 (local address 127.22.177.2:34381)
0811 10:07:24.381576 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:24.381649 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:24.382273 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:24.382549 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:24.382554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:24.383464 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:24.383820 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:24.383828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.384682 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.384683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:24.386292 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:24.386297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.386893 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.386894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:24.387030 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.387870 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:24.387879 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:24.390092 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:24.391941 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:24.391950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:24.391953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:24.392375 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:24.392597 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:24.392598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:24.392598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:24.392793 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:24.393170 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:24.393179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:24.393391 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 10:07:24.393532 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 10:07:24.393805 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":90,"threads_started":1}
I20260811 10:07:24.394843 26929 heartbeater.cc:347] Connected to a master server at 127.22.177.62:46479
I20260811 10:07:24.394987 26929 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:24.395190 26929 heartbeater.cc:511] Master 127.22.177.62:46479 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:24.395673 26564 ts_manager.cc:194] Registered new tserver with Master: e8cf10d00e9e47439f8e1e76959f9e06 (127.22.177.2:39843)
I20260811 10:07:24.396073 26564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:34381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:24.417920 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46479
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:24.503484 26934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:24.503739 26934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:24.503782 26934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:24.504981 26934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:24.505062 26934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:24.505105 26934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:24.506901 26934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:24.507189 26934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:24.507458 26934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:24.512905 26942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:24.512890 26941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:24.512871 26944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:24.512960 26934 server_base.cc:1034] running on GCE node
I20260811 10:07:24.513337 26934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:24.513635 26934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:24.514899 26934 hybrid_clock.cc:648] HybridClock initialized: now 1786442844514881 us; error 33 us; skew 500 ppm
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:24.518278 26934 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:24.518824 26934 webserver.cc:460] Webserver started at http://127.22.177.3:38689/ using document root <none> and password file <none>
I20260811 10:07:24.519070 26934 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:24.519132 26934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:24.519250 26934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:24.520411 26934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "65e02a07d9b04fc6b111ed1800e96b53"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "a2686ea09de823a227c05eb49f4c92b0"
server_key_iv: "96ac3caf8fd058f8a8d0fbd0f1dbdc9b"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.520763 26934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "65e02a07d9b04fc6b111ed1800e96b53"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "a2686ea09de823a227c05eb49f4c92b0"
server_key_iv: "96ac3caf8fd058f8a8d0fbd0f1dbdc9b"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.522413 26934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:24.523381 26951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.523605 26934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:24.523711 26934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "65e02a07d9b04fc6b111ed1800e96b53"
format_stamp: "Formatted at 2026-08-11 10:07:24 on dist-test-slave-g350"
server_key: "a2686ea09de823a227c05eb49f4c92b0"
server_key_iv: "96ac3caf8fd058f8a8d0fbd0f1dbdc9b"
server_key_version: "encryptionkey@0"
I20260811 10:07:24.523793 26934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:24.531553 26934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:24.534026 26934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:24.534236 26934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:24.534543 26934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:24.534922 26934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:24.534986 26934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.535037 26934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:24.535068 26934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:24.541792 26934 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:38735
I20260811 10:07:24.541888 27075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:38735 every 8 connection(s)
I20260811 10:07:24.542279 26934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:24.544030 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 26934
I20260811 10:07:24.544153 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:24.544334 23236 external_mini_cluster.cc:1428] Setting key 8842448ab7c209880dea749eb566b89a
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:24.553282 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.543265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:39687 (local address 127.22.177.62:46479)
0811 10:07:24.543344 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:24.543346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:24.543588 (+ 242us) server_negotiation.cc:407] Connection header received
0811 10:07:24.544197 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:24.544199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:24.544214 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:24.544257 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:24.545579 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.546228 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.547308 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.547539 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.547622 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.550057 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:24.550059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:24.550059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:24.550068 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:24.551573 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.552145 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.552147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.552147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.552228 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.552520 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.552522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.552522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.552660 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:24.552747 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:24.553075 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 10:07:24.553183 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:07:24.553509 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.543165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:46479 (local address 127.22.177.3:39687)
0811 10:07:24.543431 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:24.543487 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:24.544067 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:24.544353 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:24.544356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:24.545201 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:24.545479 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:24.545485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.546343 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.546344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:24.547208 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:24.547210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.547630 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.547632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:24.547780 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.548387 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:24.548396 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:24.549947 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:24.551764 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:24.551773 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:24.551775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:24.552051 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:24.552318 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:24.552319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:24.552319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:24.552442 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:24.552737 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:24.552746 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:24.552921 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 10:07:24.553045 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 10:07:24.553229 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260811 10:07:24.554147 27076 heartbeater.cc:347] Connected to a master server at 127.22.177.62:46479
I20260811 10:07:24.554335 27076 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:24.554574 27076 heartbeater.cc:511] Master 127.22.177.62:46479 requested a full tablet report, sending...
I20260811 10:07:24.555145 26564 ts_manager.cc:194] Registered new tserver with Master: 65e02a07d9b04fc6b111ed1800e96b53 (127.22.177.3:38735)
I20260811 10:07:24.555565 26564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:39687
I20260811 10:07:24.556658 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[26515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:24.585160 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:24.576555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.22.177.62:46479)
0811 10:07:24.576806 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:24.576808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:24.576879 (+ 71us) server_negotiation.cc:407] Connection header received
0811 10:07:24.576984 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:24.576985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:24.577003 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:24.577072 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:24.577815 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.578329 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.579122 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:24.579326 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:24.579418 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:24.582148 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:24.582154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:24.582155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:24.582163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:24.583857 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.584297 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.584299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.584299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.584386 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:24.584623 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:24.584624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:24.584624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:24.584798 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:24.584883 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:24.585016 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 10:07:24.585090 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
W20260811 10:07:24.585467 26564 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42750
I20260811 10:07:24.586117 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26634
I20260811 10:07:24.594642 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26779
I20260811 10:07:24.604846 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26934
I20260811 10:07:24.612717 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 26529
2026-08-11T10:07:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2019 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[27089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[27089](info): set up 2 sockets
Aug 11 10:07:24 dist-test-slave-g350 krb5kdc[27089](info): commencing operation
krb5kdc: starting...
W20260811 10:07:25.722416 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:25 dist-test-slave-g350 krb5kdc[27089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:25.866529 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:43757
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:43757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:25.962725 27102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:25.962970 27102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:25.962996 27102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:25.964259 27102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:25.964315 27102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:25.964329 27102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:25.964344 27102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:25.964355 27102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:25.966104 27102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:43757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:43757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:25.966400 27102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:25.966675 27102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:25.972627 27111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:25.972627 27108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:25.972642 27107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:25.973632 27102 server_base.cc:1034] running on GCE node
I20260811 10:07:25.973953 27102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:25.974344 27102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:25.975533 27102 hybrid_clock.cc:648] HybridClock initialized: now 1786442845975510 us; error 45 us; skew 500 ppm
Aug 11 10:07:25 dist-test-slave-g350 krb5kdc[27089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442845, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:25.978726 27102 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:25.979367 27102 webserver.cc:460] Webserver started at http://127.22.177.62:34879/ using document root <none> and password file <none>
I20260811 10:07:25.979671 27102 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:25.979751 27102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:25.979931 27102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:25.981179 27102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "483af2f699fa4931b2dc13bbe7855951"
format_stamp: "Formatted at 2026-08-11 10:07:25 on dist-test-slave-g350"
server_key: "19fd348e2be1452aa767a73a097e9dd6"
server_key_iv: "c3e8f1117c790a5a7f82584b1047bc7b"
server_key_version: "encryptionkey@0"
I20260811 10:07:25.981685 27102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "483af2f699fa4931b2dc13bbe7855951"
format_stamp: "Formatted at 2026-08-11 10:07:25 on dist-test-slave-g350"
server_key: "19fd348e2be1452aa767a73a097e9dd6"
server_key_iv: "c3e8f1117c790a5a7f82584b1047bc7b"
server_key_version: "encryptionkey@0"
I20260811 10:07:25.983479 27102 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 10:07:25.984733 27119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:25.985023 27102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:25.985165 27102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "483af2f699fa4931b2dc13bbe7855951"
format_stamp: "Formatted at 2026-08-11 10:07:25 on dist-test-slave-g350"
server_key: "19fd348e2be1452aa767a73a097e9dd6"
server_key_iv: "c3e8f1117c790a5a7f82584b1047bc7b"
server_key_version: "encryptionkey@0"
I20260811 10:07:25.985296 27102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:26.003161 27102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:26.005730 27102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:26.005960 27102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:26.010975 27102 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:43757
I20260811 10:07:26.011009 27178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:43757 every 8 connection(s)
I20260811 10:07:26.011605 27102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:26.012400 27179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:26.013867 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27102
I20260811 10:07:26.013955 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:26.014142 23236 external_mini_cluster.cc:1428] Setting key 33d71ea401cb6f008d4d8d102354b7fc
I20260811 10:07:26.016431 27179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951: Bootstrap starting.
I20260811 10:07:26.017351 27179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:26.017810 27179 log.cc:824] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:26.018720 27179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951: No bootstrap required, opened a new log
I20260811 10:07:26.021760 27179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483af2f699fa4931b2dc13bbe7855951" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43757 } }
I20260811 10:07:26.022007 27179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:26.022048 27179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483af2f699fa4931b2dc13bbe7855951, State: Initialized, Role: FOLLOWER
I20260811 10:07:26.022204 27179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483af2f699fa4931b2dc13bbe7855951" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43757 } }
I20260811 10:07:26.022321 27179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:26.022367 27179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:26.022434 27179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:26.023353 27179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483af2f699fa4931b2dc13bbe7855951" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43757 } }
I20260811 10:07:26.023501 27179 leader_election.cc:302] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 483af2f699fa4931b2dc13bbe7855951; no voters:
I20260811 10:07:26.023721 27179 leader_election.cc:288] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:26.023805 27185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:26.024015 27185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [term 1 LEADER]: Becoming Leader. State: Replica: 483af2f699fa4931b2dc13bbe7855951, State: Running, Role: LEADER
I20260811 10:07:26.024111 27179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:26.024111 27185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483af2f699fa4931b2dc13bbe7855951" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43757 } }
I20260811 10:07:26.024632 27186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "483af2f699fa4931b2dc13bbe7855951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483af2f699fa4931b2dc13bbe7855951" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43757 } } }
I20260811 10:07:26.024710 27186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:26.024886 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 483af2f699fa4931b2dc13bbe7855951. Latest consensus state: current_term: 1 leader_uuid: "483af2f699fa4931b2dc13bbe7855951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483af2f699fa4931b2dc13bbe7855951" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43757 } } }
I20260811 10:07:26.024950 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:26.025089 27192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:26.025740 27192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:26.027665 27192 catalog_manager.cc:1347] Generated new cluster ID: 9389bfe8444643b797f73e42bf17f614
I20260811 10:07:26.027722 27192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:26.028519 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.015100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.22.177.62:43757)
0811 10:07:26.015326 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:26.015343 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:26.015373 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:07:26.016123 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:26.016132 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:26.016191 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:26.016328 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:26.017293 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.018529 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.019773 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.020254 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.020415 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.023842 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:26.023847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:26.023847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:26.023862 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:26.026139 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.026639 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.026643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.026644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.026776 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.027067 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.027068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.027069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.027408 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:26.027580 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:26.028120 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 10:07:26.028290 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":135,"threads_started":1}
I20260811 10:07:26.036993 27192 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:26.037741 27192 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:26.043264 27192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 483af2f699fa4931b2dc13bbe7855951: Generated new TSK 0
I20260811 10:07:26.043567 27192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:26.081792 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43757
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:26.169866 27215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:26.170091 27215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:26.170118 27215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:26.171272 27215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:26.171314 27215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:26.171341 27215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:26.172930 27215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:26.173151 27215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:26.173350 27215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:26.179545 27215 server_base.cc:1034] running on GCE node
W20260811 10:07:26.179515 27222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:26.179522 27221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:26.179522 27224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:26.180171 27215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:26.180451 27215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:26.181680 27215 hybrid_clock.cc:648] HybridClock initialized: now 1786442846181651 us; error 55 us; skew 500 ppm
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:26.184926 27215 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:26.185448 27215 webserver.cc:460] Webserver started at http://127.22.177.1:46105/ using document root <none> and password file <none>
I20260811 10:07:26.185760 27215 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:26.185868 27215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:26.186048 27215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:26.187227 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "cfc1d436e7c14fa6a4c25221a5c161ed"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "53b8d3abc72fa02a62d68569366a676a"
server_key_iv: "0600d081efe1ebbd09ee0614763e04f4"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.187680 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "cfc1d436e7c14fa6a4c25221a5c161ed"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "53b8d3abc72fa02a62d68569366a676a"
server_key_iv: "0600d081efe1ebbd09ee0614763e04f4"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.189218 27215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:26.190164 27231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.190475 27215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:26.190598 27215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "cfc1d436e7c14fa6a4c25221a5c161ed"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "53b8d3abc72fa02a62d68569366a676a"
server_key_iv: "0600d081efe1ebbd09ee0614763e04f4"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.190698 27215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:26.202872 27215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:26.205864 27215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:26.206059 27215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:26.206349 27215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:26.206702 27215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:26.206763 27215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.206807 27215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:26.206836 27215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.212992 27215 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:37513
I20260811 10:07:26.213044 27359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:37513 every 8 connection(s)
I20260811 10:07:26.213382 27215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:26.218173 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27215
I20260811 10:07:26.218326 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:26.218518 23236 external_mini_cluster.cc:1428] Setting key 7992f981ed058a0048fcaf431c404d40
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:26.224975 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.214242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:55685 (local address 127.22.177.62:43757)
0811 10:07:26.214327 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:26.214329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:26.214670 (+ 341us) server_negotiation.cc:407] Connection header received
0811 10:07:26.215732 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:26.215734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:26.215745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:26.215789 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:26.216544 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.217077 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.218290 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.218623 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.218709 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.221878 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:26.221879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:26.221879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:26.221887 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:26.223236 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.223807 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.223809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.223809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.223866 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.224230 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.224232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.224232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.224385 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:26.224474 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:26.224772 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 10:07:26.224881 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:07:26.225451 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.214188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:43757 (local address 127.22.177.1:55685)
0811 10:07:26.214430 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:26.214563 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:26.215615 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:26.215852 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:26.215855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:26.216162 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:26.216453 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:26.216459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.217182 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.217183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:26.218173 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:26.218176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.218847 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.218849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:26.218996 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.219681 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:26.219690 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:26.221772 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:26.223332 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:26.223340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:26.223343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:26.223692 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:26.224016 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:26.224017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:26.224017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:26.224168 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:26.224464 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:26.224471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:26.224628 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 10:07:26.224738 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 10:07:26.224921 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
I20260811 10:07:26.226122 27360 heartbeater.cc:347] Connected to a master server at 127.22.177.62:43757
I20260811 10:07:26.226250 27360 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:26.226485 27360 heartbeater.cc:511] Master 127.22.177.62:43757 requested a full tablet report, sending...
I20260811 10:07:26.227178 27136 ts_manager.cc:194] Registered new tserver with Master: cfc1d436e7c14fa6a4c25221a5c161ed (127.22.177.1:37513)
I20260811 10:07:26.228158 27136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:55685
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:26.249234 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43757
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:26.337291 27365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:26.337517 27365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:26.337541 27365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:26.338850 27365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:26.338903 27365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:26.338932 27365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:26.340564 27365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:26.340842 27365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:26.341097 27365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:26.346930 27373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:26.346937 27370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:26.346993 27371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:26.347026 27365 server_base.cc:1034] running on GCE node
I20260811 10:07:26.347445 27365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:26.347734 27365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:26.348910 27365 hybrid_clock.cc:648] HybridClock initialized: now 1786442846348881 us; error 45 us; skew 500 ppm
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:26.352020 27365 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:26.352591 27365 webserver.cc:460] Webserver started at http://127.22.177.2:41883/ using document root <none> and password file <none>
I20260811 10:07:26.352843 27365 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:26.352917 27365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:26.353077 27365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:26.354285 27365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "6e86c60902834eeabd926716c563c521"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "dff32f6e6181e40e7894325c55b5a6d4"
server_key_iv: "797e4adf2b0cff495f10dd93073ab2bd"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.354681 27365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "6e86c60902834eeabd926716c563c521"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "dff32f6e6181e40e7894325c55b5a6d4"
server_key_iv: "797e4adf2b0cff495f10dd93073ab2bd"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.356253 27365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:07:26.357265 27381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.357566 27365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:26.357760 27365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "6e86c60902834eeabd926716c563c521"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "dff32f6e6181e40e7894325c55b5a6d4"
server_key_iv: "797e4adf2b0cff495f10dd93073ab2bd"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.357863 27365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:26.374375 27365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:26.377111 27365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:26.377311 27365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:26.377646 27365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:26.378028 27365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:26.378086 27365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.378124 27365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:26.378151 27365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.384816 27365 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:40465
I20260811 10:07:26.384927 27503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:40465 every 8 connection(s)
I20260811 10:07:26.385283 27365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:26.385987 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27365
I20260811 10:07:26.386080 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:26.386341 23236 external_mini_cluster.cc:1428] Setting key f5d905444babce2452be18767f9f8cfe
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:26.397996 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.386213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:43757 (local address 127.22.177.2:55163)
0811 10:07:26.386384 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:26.386429 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:26.387517 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:26.387783 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:26.387787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:26.388064 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:26.388377 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:26.388382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.389355 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.389357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:26.390863 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:26.390867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.391283 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.391285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:26.391460 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.392102 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:26.392114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:26.394186 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:26.396074 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:26.396085 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:26.396088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:26.396457 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:26.396682 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:26.396683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:26.396684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:26.396846 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:26.397182 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:26.397190 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:26.397401 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 10:07:26.397542 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 10:07:26.397793 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260811 10:07:26.398011 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.386225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:55163 (local address 127.22.177.62:43757)
0811 10:07:26.386304 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:26.386307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:26.386537 (+ 230us) server_negotiation.cc:407] Connection header received
0811 10:07:26.387631 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:26.387633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:26.387646 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:26.387721 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:26.388466 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.389211 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.390978 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.391170 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.391243 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.394375 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:26.394377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:26.394377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:26.394384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:26.395970 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.396551 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.396552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.396552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.396607 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.396924 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.396924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.396925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.397041 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:26.397120 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:26.397857 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 10:07:26.397945 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:07:26.398603 27505 heartbeater.cc:347] Connected to a master server at 127.22.177.62:43757
I20260811 10:07:26.398738 27505 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:26.398993 27505 heartbeater.cc:511] Master 127.22.177.62:43757 requested a full tablet report, sending...
I20260811 10:07:26.399549 27136 ts_manager.cc:194] Registered new tserver with Master: 6e86c60902834eeabd926716c563c521 (127.22.177.2:40465)
I20260811 10:07:26.399971 27136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:55163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:26.421018 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43757
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:26.506649 27510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:26.506893 27510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:26.506922 27510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:26.508217 27510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:26.508285 27510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:26.508315 27510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:26.509987 27510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:26.510212 27510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:26.510419 27510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:26.515995 27515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:26.516004 27518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:26.516067 27516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:26.516299 27510 server_base.cc:1034] running on GCE node
I20260811 10:07:26.516566 27510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:26.516887 27510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:26.518091 27510 hybrid_clock.cc:648] HybridClock initialized: now 1786442846518083 us; error 36 us; skew 500 ppm
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:26.520998 27510 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:26.521582 27510 webserver.cc:460] Webserver started at http://127.22.177.3:35753/ using document root <none> and password file <none>
I20260811 10:07:26.521906 27510 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:26.521977 27510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:26.522135 27510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:26.523233 27510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "4757c069ed244e46a96a40f2bbad07e0"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "3b0acc8aa02875a7e010d1e740a750dc"
server_key_iv: "634354730f0d393d7e389c5ccbacf53a"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.523608 27510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "4757c069ed244e46a96a40f2bbad07e0"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "3b0acc8aa02875a7e010d1e740a750dc"
server_key_iv: "634354730f0d393d7e389c5ccbacf53a"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.525291 27510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:26.526305 27526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.526484 27510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:26.526578 27510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "4757c069ed244e46a96a40f2bbad07e0"
format_stamp: "Formatted at 2026-08-11 10:07:26 on dist-test-slave-g350"
server_key: "3b0acc8aa02875a7e010d1e740a750dc"
server_key_iv: "634354730f0d393d7e389c5ccbacf53a"
server_key_version: "encryptionkey@0"
I20260811 10:07:26.526641 27510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:26.536882 27510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:26.539492 27510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:26.539623 27510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:26.539832 27510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:26.540169 27510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:26.540203 27510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.540227 27510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:26.540278 27510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:26.546947 27510 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:33757
I20260811 10:07:26.546988 27648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:33757 every 8 connection(s)
I20260811 10:07:26.547386 27510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:26.557582 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27510
I20260811 10:07:26.557729 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:26.557962 23236 external_mini_cluster.cc:1428] Setting key 1120e6a08a025f8dca3afbcd6a8d7af6
I20260811 10:07:26.558077 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.548266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:41585 (local address 127.22.177.62:43757)
0811 10:07:26.548359 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:26.548362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:26.548527 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:07:26.549486 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:26.549487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:26.549497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:26.549554 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:26.550336 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.550832 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.552141 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.552344 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.552403 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.554852 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:26.554853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:26.554854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:26.554861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:26.556407 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.556868 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.556869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.556870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.556928 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.557208 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.557208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.557208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.557316 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:26.557462 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:26.557889 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 10:07:26.557980 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 10:07:26.558295 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.548204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:43757 (local address 127.22.177.3:41585)
0811 10:07:26.548399 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:26.548439 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:26.549383 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:26.549736 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:26.549740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:26.550048 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:26.550245 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:26.550249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.550939 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.550940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:26.552040 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:26.552043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.552445 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.552446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:26.552556 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.553108 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:26.553117 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:26.554724 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:26.556497 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:26.556504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:26.556507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:26.556773 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:26.557015 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:26.557016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:26.557017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:26.557142 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:26.557457 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:26.557464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:26.557651 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 10:07:26.557863 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 10:07:26.558044 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260811 10:07:26.558825 27649 heartbeater.cc:347] Connected to a master server at 127.22.177.62:43757
I20260811 10:07:26.558936 27649 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:26.559137 27649 heartbeater.cc:511] Master 127.22.177.62:43757 requested a full tablet report, sending...
I20260811 10:07:26.559574 27136 ts_manager.cc:194] Registered new tserver with Master: 4757c069ed244e46a96a40f2bbad07e0 (127.22.177.3:33757)
I20260811 10:07:26.559962 27136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:41585
I20260811 10:07:26.570050 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:26.596585 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.587791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.22.177.1:37513)
0811 10:07:26.587976 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:26.587981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:26.587991 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:26.588039 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:26.588041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:26.588056 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:26.588151 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:26.588882 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.589684 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.590433 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.590773 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.590865 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.593391 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:26.593402 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:26.593403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:26.593418 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:26.595252 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.595649 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.595652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.595653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.595720 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.595955 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.595956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.595956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.596115 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:26.596309 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:26.596356 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 10:07:26.596447 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:26.605934 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:26.598120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.22.177.62:43757)
0811 10:07:26.598203 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:26.598206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:26.598242 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:07:26.598328 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:26.598329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:26.598341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:26.598405 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:26.598928 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.599518 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.600626 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:26.600843 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:26.600935 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:26.603422 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:26.603424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:26.603424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:26.603431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:26.604708 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.605043 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.605044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.605045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.605096 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:26.605334 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:26.605334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:26.605335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:26.605484 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:26.605561 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:26.605779 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 10:07:26.605865 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 10:07:26.606199 27136 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50874
I20260811 10:07:26.606597 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27215
I20260811 10:07:26.614168 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27365
I20260811 10:07:26.624445 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27510
I20260811 10:07:26.632165 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27102
2026-08-11T10:07:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2019 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27663](info): set up 2 sockets
Aug 11 10:07:26 dist-test-slave-g350 krb5kdc[27663](info): commencing operation
krb5kdc: starting...
W20260811 10:07:27.777311 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:27 dist-test-slave-g350 krb5kdc[27663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:27.917389 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:33449
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:33449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:28.011268 27676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:28.011554 27676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:28.011596 27676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:28.012837 27676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:28.012923 27676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:28.012950 27676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:28.012982 27676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:28.013010 27676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:28.014783 27676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:33449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:33449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:28.015103 27676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:28.015357 27676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:28.020830 27681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.020901 27682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:28.020932 27676 server_base.cc:1034] running on GCE node
W20260811 10:07:28.021061 27685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:28.021405 27676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:28.021811 27676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:28.023011 27676 hybrid_clock.cc:648] HybridClock initialized: now 1786442848022999 us; error 30 us; skew 500 ppm
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:28.026404 27676 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:28.026846 27676 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:28.026929 27676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:28.027088 27676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:28.028228 27676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "7d481edc736543fba26f4774f6468078"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "3d2fdee91b3bb4c65b7241ff90f7b2c6"
server_key_iv: "a09fb9ee23a1ec0ee01944aa1b3ad5db"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.028650 27676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "7d481edc736543fba26f4774f6468078"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "3d2fdee91b3bb4c65b7241ff90f7b2c6"
server_key_iv: "a09fb9ee23a1ec0ee01944aa1b3ad5db"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.030472 27676 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 10:07:28.031409 27691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.031638 27676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:28.031764 27676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "7d481edc736543fba26f4774f6468078"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "3d2fdee91b3bb4c65b7241ff90f7b2c6"
server_key_iv: "a09fb9ee23a1ec0ee01944aa1b3ad5db"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.031863 27676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:28.053431 27676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:28.056294 27676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:28.056488 27676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:28.061069 27676 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:33449
I20260811 10:07:28.061096 27751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:33449 every 8 connection(s)
I20260811 10:07:28.061434 27676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:28.062556 27752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:28.064168 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27676
I20260811 10:07:28.064306 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:28.064496 23236 external_mini_cluster.cc:1428] Setting key 1705f4c331119eec71586bd5badd98ec
I20260811 10:07:28.065701 27752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078: Bootstrap starting.
I20260811 10:07:28.066506 27752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:28.066852 27752 log.cc:824] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:28.067579 27752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078: No bootstrap required, opened a new log
I20260811 10:07:28.069504 27752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d481edc736543fba26f4774f6468078" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33449 } }
I20260811 10:07:28.069744 27752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:28.069799 27752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d481edc736543fba26f4774f6468078, State: Initialized, Role: FOLLOWER
I20260811 10:07:28.069947 27752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d481edc736543fba26f4774f6468078" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33449 } }
I20260811 10:07:28.070035 27752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:28.070092 27752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:28.070142 27752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:28.070760 27752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d481edc736543fba26f4774f6468078" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33449 } }
I20260811 10:07:28.070892 27752 leader_election.cc:302] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d481edc736543fba26f4774f6468078; no voters:
I20260811 10:07:28.071133 27752 leader_election.cc:288] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:28.071163 27757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:28.071354 27757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [term 1 LEADER]: Becoming Leader. State: Replica: 7d481edc736543fba26f4774f6468078, State: Running, Role: LEADER
I20260811 10:07:28.071434 27752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:28.071445 27757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d481edc736543fba26f4774f6468078" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33449 } }
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:28.071959 27759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d481edc736543fba26f4774f6468078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d481edc736543fba26f4774f6468078" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33449 } } }
I20260811 10:07:28.072057 27759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:28.072238 27758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d481edc736543fba26f4774f6468078. Latest consensus state: current_term: 1 leader_uuid: "7d481edc736543fba26f4774f6468078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d481edc736543fba26f4774f6468078" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 33449 } } }
I20260811 10:07:28.072317 27758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:28.072296 27770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:28.073415 27770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:28.076427 27770 catalog_manager.cc:1347] Generated new cluster ID: e937c5461ccb4b3191316178231d6c58
I20260811 10:07:28.076488 27770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:28.077080 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.065128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.22.177.62:33449)
0811 10:07:28.065381 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.065391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.065423 (+ 32us) server_negotiation.cc:407] Connection header received
0811 10:07:28.066066 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.066077 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.066151 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.066323 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:28.067075 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.068094 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.068925 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.069372 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.069502 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.072863 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:28.072868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:28.072869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:28.072884 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:28.074909 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.075404 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.075408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.075409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.075527 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.075858 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.075872 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.075872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.076205 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:28.076354 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.076628 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.076839 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":141,"threads_started":1}
I20260811 10:07:28.114799 27770 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:28.115628 27770 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:28.128582 27770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078: Generated new TSK 0
I20260811 10:07:28.128849 27770 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:28.172888 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33449
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:28.259428 27785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:28.259656 27785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:28.259684 27785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:28.260937 27785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:28.260982 27785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:28.261010 27785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:28.262646 27785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:28.262935 27785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:28.263199 27785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:28.268666 27790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.268666 27791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.268669 27793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:28.269058 27785 server_base.cc:1034] running on GCE node
I20260811 10:07:28.269315 27785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:28.269585 27785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:28.270810 27785 hybrid_clock.cc:648] HybridClock initialized: now 1786442848270802 us; error 25 us; skew 500 ppm
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:28.274147 27785 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:28.274601 27785 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:28.274673 27785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:28.274834 27785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:28.276154 27785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "f24ddc26670c452ca4569399079eda92"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "7ed7c2e86afb52eac6093dc7a00420d4"
server_key_iv: "70b968a26231c0f1c0809747dd173a08"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.276566 27785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "f24ddc26670c452ca4569399079eda92"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "7ed7c2e86afb52eac6093dc7a00420d4"
server_key_iv: "70b968a26231c0f1c0809747dd173a08"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.278200 27785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:28.279140 27800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.279402 27785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:28.279503 27785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "f24ddc26670c452ca4569399079eda92"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "7ed7c2e86afb52eac6093dc7a00420d4"
server_key_iv: "70b968a26231c0f1c0809747dd173a08"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.279580 27785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:28.308785 27785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:28.311338 27785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:28.311466 27785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:28.311791 27785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:28.312140 27785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:28.312197 27785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.312234 27785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:28.312263 27785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.318770 27785 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:36781
I20260811 10:07:28.318809 27924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:36781 every 8 connection(s)
I20260811 10:07:28.319100 27785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:28.319738 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27785
I20260811 10:07:28.319916 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:28.320139 23236 external_mini_cluster.cc:1428] Setting key 54fde8c240d178c0ec2317ed8a2e0afe
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
I20260811 10:07:28.330356 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.319964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:57401 (local address 127.22.177.62:33449)
0811 10:07:28.320019 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.320021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.320282 (+ 261us) server_negotiation.cc:407] Connection header received
0811 10:07:28.320940 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.320941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.320953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.320999 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:28.322106 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.322780 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.324385 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.324586 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.324645 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.327380 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:28.327381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:28.327382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:28.327390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:28.328852 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.329292 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.329293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.329293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.329362 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.329649 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.329650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.329651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.329781 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:28.329865 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.330157 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.330251 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:07:28.330543 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.319926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:33449 (local address 127.22.177.1:57401)
0811 10:07:28.320119 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.320158 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.320837 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.321068 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.321072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.321366 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:28.322003 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.322009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.322918 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.322919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.324240 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.324243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.324684 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.324685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.324864 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.325468 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:28.325480 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:28.327278 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:28.328958 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:28.328966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:28.328969 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:28.329212 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:28.329436 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:28.329437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:28.329437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:28.329558 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:28.329857 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:28.329865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:28.330013 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.330121 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.330294 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260811 10:07:28.331142 27925 heartbeater.cc:347] Connected to a master server at 127.22.177.62:33449
I20260811 10:07:28.331342 27925 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:28.331557 27925 heartbeater.cc:511] Master 127.22.177.62:33449 requested a full tablet report, sending...
I20260811 10:07:28.332142 27711 ts_manager.cc:194] Registered new tserver with Master: f24ddc26670c452ca4569399079eda92 (127.22.177.1:36781)
I20260811 10:07:28.332860 27711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:57401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:28.353750 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33449
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:28.440459 27930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:28.440691 27930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:28.440719 27930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:28.441924 27930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:28.441974 27930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:28.442004 27930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:28.443506 27930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:28.443717 27930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:28.443904 27930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:28.449508 27938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.449544 27936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.449563 27935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:28.449546 27930 server_base.cc:1034] running on GCE node
I20260811 10:07:28.449970 27930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:28.450227 27930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:28.451408 27930 hybrid_clock.cc:648] HybridClock initialized: now 1786442848451396 us; error 32 us; skew 500 ppm
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:28.454267 27930 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:28.454737 27930 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:28.454793 27930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:28.454967 27930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:28.456151 27930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "78b532ff1c354153b2937701392e09cb"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "c8cef46b137fa9dbd2c4d5a62bc4bf88"
server_key_iv: "7f80eada6705fd153ce45ffe914c2274"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.456564 27930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "78b532ff1c354153b2937701392e09cb"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "c8cef46b137fa9dbd2c4d5a62bc4bf88"
server_key_iv: "7f80eada6705fd153ce45ffe914c2274"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.458060 27930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:28.458921 27944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.459236 27930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:28.459383 27930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "78b532ff1c354153b2937701392e09cb"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "c8cef46b137fa9dbd2c4d5a62bc4bf88"
server_key_iv: "7f80eada6705fd153ce45ffe914c2274"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.459463 27930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:28.490010 27930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:28.492427 27930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:28.492620 27930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:28.492928 27930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:28.493279 27930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:28.493335 27930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.493376 27930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:28.493429 27930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.499675 27930 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:40069
I20260811 10:07:28.499733 28069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:40069 every 8 connection(s)
I20260811 10:07:28.500006 27930 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:28.500391 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 27930
I20260811 10:07:28.500504 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:28.500746 23236 external_mini_cluster.cc:1428] Setting key e2e4de41395583f1f8eeff8c01ee95a2
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:28.511281 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.500925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:49209 (local address 127.22.177.62:33449)
0811 10:07:28.500994 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.500997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.501294 (+ 297us) server_negotiation.cc:407] Connection header received
0811 10:07:28.501952 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.501954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.501967 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.502041 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:28.503179 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.503817 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.505121 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.505339 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.505403 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.508270 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:28.508272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:28.508273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:28.508281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:28.509800 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.510251 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.510252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.510253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.510335 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.510583 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.510585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.510585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.510715 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:28.510798 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.511086 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.511170 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:07:28.511518 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.500894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:33449 (local address 127.22.177.2:49209)
0811 10:07:28.501086 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.501140 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.501840 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.502115 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.502119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.502554 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:28.503069 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.503074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.503940 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.503941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.505021 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.505024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.505435 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.505437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.505609 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.506361 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:28.506372 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:28.508159 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:28.509923 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:28.509931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:28.509933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:28.510166 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:28.510395 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:28.510396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:28.510397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:28.510515 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:28.510788 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:28.510796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:28.510949 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.511065 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.511251 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260811 10:07:28.512087 28070 heartbeater.cc:347] Connected to a master server at 127.22.177.62:33449
I20260811 10:07:28.512226 28070 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:28.512441 28070 heartbeater.cc:511] Master 127.22.177.62:33449 requested a full tablet report, sending...
I20260811 10:07:28.512892 27711 ts_manager.cc:194] Registered new tserver with Master: 78b532ff1c354153b2937701392e09cb (127.22.177.2:40069)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
I20260811 10:07:28.513464 27711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:49209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:28.532832 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33449
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:28.621017 28075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:28.621294 28075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:28.621335 28075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:28.622620 28075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:28.622702 28075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:28.622750 28075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:28.624338 28075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:33449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:28.624624 28075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:28.624873 28075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:28.630699 28082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.630684 28081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.630684 28085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:28.631259 28075 server_base.cc:1034] running on GCE node
I20260811 10:07:28.631551 28075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:28.631816 28075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:28.632998 28075 hybrid_clock.cc:648] HybridClock initialized: now 1786442848632978 us; error 28 us; skew 500 ppm
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:28.635737 28075 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:28.636164 28075 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:28.636241 28075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:28.636448 28075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:28.637579 28075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "3df9526891004ee8ab22bc279791c110"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "86e5ae6e69789a90c04734377a3dbf41"
server_key_iv: "46e42b71f5146f715bb2bc3b9d0f5467"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.638029 28075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "3df9526891004ee8ab22bc279791c110"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "86e5ae6e69789a90c04734377a3dbf41"
server_key_iv: "46e42b71f5146f715bb2bc3b9d0f5467"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.639460 28075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:28.640228 28091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.640409 28075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:28.640522 28075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "3df9526891004ee8ab22bc279791c110"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "86e5ae6e69789a90c04734377a3dbf41"
server_key_iv: "46e42b71f5146f715bb2bc3b9d0f5467"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.640615 28075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:28.655125 28075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:28.657851 28075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:28.657975 28075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:28.658181 28075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:28.658533 28075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:28.658569 28075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.658594 28075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:28.658638 28075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.665163 28075 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:36627
I20260811 10:07:28.665197 28214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:36627 every 8 connection(s)
I20260811 10:07:28.665500 28075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:28.669479 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28075
I20260811 10:07:28.669623 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:28.669818 23236 external_mini_cluster.cc:1428] Setting key accf84444352b0baea6d1e1d5017956b
Aug 11 10:07:28 dist-test-slave-g350 krb5kdc[27663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442848, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:28.675410 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.666228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:40621 (local address 127.22.177.62:33449)
0811 10:07:28.666334 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.666337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.666528 (+ 191us) server_negotiation.cc:407] Connection header received
0811 10:07:28.667073 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.667075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.667089 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.667171 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:28.668168 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.668689 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.670052 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.670309 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.670397 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.672783 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:28.672784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:28.672785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:28.672792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:28.674139 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.674578 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.674579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.674580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.674632 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.674853 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.674854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.674854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.674956 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:28.675006 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.675242 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.675343 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:07:28.675658 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.666206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:33449 (local address 127.22.177.3:40621)
0811 10:07:28.666399 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.666448 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.666957 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.667252 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.667257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.667549 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:28.668062 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.668067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.668801 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.668802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.669956 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.669959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.670411 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.670412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.670543 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.671107 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:28.671116 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:28.672696 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:28.674222 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:28.674229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:28.674231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:28.674511 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:28.674692 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:28.674693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:28.674693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:28.674794 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:28.675011 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:28.675016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:28.675133 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.675217 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.675389 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260811 10:07:28.676235 28215 heartbeater.cc:347] Connected to a master server at 127.22.177.62:33449
I20260811 10:07:28.676457 28215 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:28.676645 28215 heartbeater.cc:511] Master 127.22.177.62:33449 requested a full tablet report, sending...
I20260811 10:07:28.677122 27711 ts_manager.cc:194] Registered new tserver with Master: 3df9526891004ee8ab22bc279791c110 (127.22.177.3:36627)
I20260811 10:07:28.677572 27711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:40621
I20260811 10:07:28.681852 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:07:28.689904 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.683319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.22.177.62:33449)
0811 10:07:28.683407 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.683409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.683523 (+ 114us) server_negotiation.cc:407] Connection header received
0811 10:07:28.683603 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.683604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.683615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.683671 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:28.684261 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.684759 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.685487 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.685777 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.685853 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.687140 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:28.687142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:28.687142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:28.687149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:28.688451 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.688906 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.688908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.688908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.688979 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:28.689236 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:28.689237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:28.689237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:28.689419 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:28.689541 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.689712 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.689796 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:07:28.691701 27711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:28.693413 27711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:28.700873 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.696465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.22.177.1:36781)
0811 10:07:28.696667 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.696674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.696686 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:28.696835 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.696839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.696858 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.696964 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:28.697245 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.698130 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.699807 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.700388 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.700485 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.700555 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.700730 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.700773 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
I20260811 10:07:28.700860 28229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.696221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36781 (local address 127.0.0.1:49312)
0811 10:07:28.696650 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.696660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.696693 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.697082 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.697083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.697086 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:28.697142 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.697143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.698251 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.698252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.699692 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.699695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.700497 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.700498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.700588 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.700605 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.700710 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.700773 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":95,"threads_started":1}
I20260811 10:07:28.701088 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.696409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.22.177.3:36627)
0811 10:07:28.696770 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.696774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.696784 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:28.696824 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.696826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.696842 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.697076 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:28.697397 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.698492 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.699752 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.700518 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.700603 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.700675 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.700955 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.700998 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":100,"threads_started":1}
I20260811 10:07:28.701351 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.696212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.22.177.2:40069)
0811 10:07:28.696569 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.696573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.696584 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:07:28.696629 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.696631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.696649 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.696799 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:28.697085 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.698447 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.699906 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.700591 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.701138 (+ 547us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.701213 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.701252 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.701287 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":111,"threads_started":1}
I20260811 10:07:28.700997 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.696114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:40069 (local address 127.0.0.1:51382)
0811 10:07:28.696302 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.696344 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.696415 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.696780 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.696784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.696786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:28.696870 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.696871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.698578 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.698579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.699814 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.699816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.700717 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.700718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.700799 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.700804 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.700856 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.700903 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":46,"thread_start_us":65,"threads_started":1}
I20260811 10:07:28.701100 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.696153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:36627 (local address 127.0.0.1:53720)
0811 10:07:28.696476 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.696489 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.696548 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.697227 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.697228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.697232 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:28.697290 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.697292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.698620 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.698621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.699639 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.699642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.700715 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.700717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.700806 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.700810 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.700863 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.700903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260811 10:07:28.702286 28000 tablet_service.cc:1467] Processing CreateTablet for tablet a1f1eecd93eb4a9dbfdb594726b08635 (DEFAULT_TABLE table=test-table [id=58a0ad05d2944f6ca5330a8378719ca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:28.702288 27855 tablet_service.cc:1467] Processing CreateTablet for tablet a1f1eecd93eb4a9dbfdb594726b08635 (DEFAULT_TABLE table=test-table [id=58a0ad05d2944f6ca5330a8378719ca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:28.702705 28000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f1eecd93eb4a9dbfdb594726b08635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:28.702705 27855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f1eecd93eb4a9dbfdb594726b08635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:28.705271 28144 tablet_service.cc:1467] Processing CreateTablet for tablet a1f1eecd93eb4a9dbfdb594726b08635 (DEFAULT_TABLE table=test-table [id=58a0ad05d2944f6ca5330a8378719ca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:28.705727 28144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f1eecd93eb4a9dbfdb594726b08635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:28.706516 28234 tablet_bootstrap.cc:492] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92: Bootstrap starting.
I20260811 10:07:28.707295 28234 tablet_bootstrap.cc:654] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:28.708007 28234 log.cc:824] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:28.708484 28235 tablet_bootstrap.cc:492] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Bootstrap starting.
I20260811 10:07:28.709051 28234 tablet_bootstrap.cc:492] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92: No bootstrap required, opened a new log
I20260811 10:07:28.709144 28234 ts_tablet_manager.cc:1397] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:07:28.709353 28235 tablet_bootstrap.cc:654] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:28.710055 28237 tablet_bootstrap.cc:492] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Bootstrap starting.
I20260811 10:07:28.710054 28235 log.cc:824] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:28.710718 28237 tablet_bootstrap.cc:654] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:28.710918 28235 tablet_bootstrap.cc:492] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: No bootstrap required, opened a new log
I20260811 10:07:28.710994 28235 ts_tablet_manager.cc:1397] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:07:28.711344 28237 log.cc:824] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:28.711277 28234 raft_consensus.cc:348] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.711441 28234 raft_consensus.cc:374] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:28.711488 28234 raft_consensus.cc:729] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24ddc26670c452ca4569399079eda92, State: Initialized, Role: FOLLOWER
I20260811 10:07:28.711628 28234 consensus_queue.cc:260] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.711939 27925 heartbeater.cc:503] Master 127.22.177.62:33449 was elected leader, sending a full tablet report...
I20260811 10:07:28.712077 28237 tablet_bootstrap.cc:492] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: No bootstrap required, opened a new log
I20260811 10:07:28.712163 28237 ts_tablet_manager.cc:1397] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:07:28.712352 28234 ts_tablet_manager.cc:1428] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:28.713266 28235 raft_consensus.cc:348] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.713447 28235 raft_consensus.cc:374] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:28.713485 28235 raft_consensus.cc:729] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b532ff1c354153b2937701392e09cb, State: Initialized, Role: FOLLOWER
I20260811 10:07:28.713649 28235 consensus_queue.cc:260] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.713951 28235 ts_tablet_manager.cc:1428] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:28.713985 28070 heartbeater.cc:503] Master 127.22.177.62:33449 was elected leader, sending a full tablet report...
I20260811 10:07:28.714254 28237 raft_consensus.cc:348] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.714407 28237 raft_consensus.cc:374] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:28.714452 28237 raft_consensus.cc:729] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df9526891004ee8ab22bc279791c110, State: Initialized, Role: FOLLOWER
I20260811 10:07:28.714565 28237 consensus_queue.cc:260] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.714814 28241 raft_consensus.cc:482] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:28.714900 28237 ts_tablet_manager.cc:1428] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:28.714893 28241 raft_consensus.cc:504] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.715026 28215 heartbeater.cc:503] Master 127.22.177.62:33449 was elected leader, sending a full tablet report...
I20260811 10:07:28.715240 28241 leader_election.cc:288] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24ddc26670c452ca4569399079eda92 (127.22.177.1:36781), 3df9526891004ee8ab22bc279791c110 (127.22.177.3:36627)
I20260811 10:07:28.718225 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.715368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36781 (local address 127.22.177.2:54509)
0811 10:07:28.715486 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.715500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.715543 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.715719 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.715720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.715721 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:28.715796 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.715797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.716557 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.716558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.717324 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.717326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.717980 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.717981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.718063 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.718068 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.718119 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.718158 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 10:07:28.718310 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.715525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:44435 (local address 127.22.177.3:36627)
0811 10:07:28.715585 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.715587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.715715 (+ 128us) server_negotiation.cc:407] Connection header received
0811 10:07:28.715815 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.715815 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.715825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.715879 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:28.716105 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.716558 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.717503 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.717933 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.718029 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.718037 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.718215 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.718252 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:07:28.718324 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.715396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:36627 (local address 127.22.177.2:44435)
0811 10:07:28.715656 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.715667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:28.715740 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:28.715944 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:28.715945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:28.715946 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:28.716003 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.716004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.716663 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.716664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.717408 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:28.717410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.718033 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.718034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:28.718122 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.718125 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:28.718206 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:07:28.718248 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
I20260811 10:07:28.718585 28166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f1eecd93eb4a9dbfdb594726b08635" candidate_uuid: "78b532ff1c354153b2937701392e09cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3df9526891004ee8ab22bc279791c110" is_pre_election: true
I20260811 10:07:28.718688 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.715442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:54509 (local address 127.22.177.1:36781)
0811 10:07:28.715621 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.715623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.715630 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:07:28.715653 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.715654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.715663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.715714 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:28.715888 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.716463 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.717417 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.717912 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.717983 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.717992 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.718576 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.718619 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260811 10:07:28.718724 28166 raft_consensus.cc:2454] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78b532ff1c354153b2937701392e09cb in term 0.
I20260811 10:07:28.718956 27947 leader_election.cc:302] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3df9526891004ee8ab22bc279791c110, 78b532ff1c354153b2937701392e09cb; no voters:
I20260811 10:07:28.718943 27877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f1eecd93eb4a9dbfdb594726b08635" candidate_uuid: "78b532ff1c354153b2937701392e09cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24ddc26670c452ca4569399079eda92" is_pre_election: true
I20260811 10:07:28.719084 27877 raft_consensus.cc:2454] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78b532ff1c354153b2937701392e09cb in term 0.
I20260811 10:07:28.719200 28241 raft_consensus.cc:2781] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:28.719269 28241 raft_consensus.cc:482] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:28.719301 28241 raft_consensus.cc:3037] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:28.719987 28241 raft_consensus.cc:504] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.720139 28241 leader_election.cc:288] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [CANDIDATE]: Term 1 election: Requested vote from peers f24ddc26670c452ca4569399079eda92 (127.22.177.1:36781), 3df9526891004ee8ab22bc279791c110 (127.22.177.3:36627)
I20260811 10:07:28.720319 27877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f1eecd93eb4a9dbfdb594726b08635" candidate_uuid: "78b532ff1c354153b2937701392e09cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24ddc26670c452ca4569399079eda92"
I20260811 10:07:28.720328 28166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f1eecd93eb4a9dbfdb594726b08635" candidate_uuid: "78b532ff1c354153b2937701392e09cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3df9526891004ee8ab22bc279791c110"
I20260811 10:07:28.720397 28166 raft_consensus.cc:3037] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:28.720396 27877 raft_consensus.cc:3037] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:28.721051 28166 raft_consensus.cc:2454] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78b532ff1c354153b2937701392e09cb in term 1.
I20260811 10:07:28.721120 27877 raft_consensus.cc:2454] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78b532ff1c354153b2937701392e09cb in term 1.
I20260811 10:07:28.721228 27947 leader_election.cc:302] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3df9526891004ee8ab22bc279791c110, 78b532ff1c354153b2937701392e09cb; no voters:
I20260811 10:07:28.721312 28241 raft_consensus.cc:2781] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:28.721505 28241 raft_consensus.cc:686] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 1 LEADER]: Becoming Leader. State: Replica: 78b532ff1c354153b2937701392e09cb, State: Running, Role: LEADER
I20260811 10:07:28.721643 28241 consensus_queue.cc:237] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } }
I20260811 10:07:28.722466 27711 catalog_manager.cc:5495] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 78b532ff1c354153b2937701392e09cb (127.22.177.2). New cstate: current_term: 1 leader_uuid: "78b532ff1c354153b2937701392e09cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78b532ff1c354153b2937701392e09cb" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 40069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:28.729229 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.726773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.22.177.2:40069)
0811 10:07:28.726835 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.726837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.726847 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:28.726919 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.726920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.726935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.726987 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:28.727141 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.727662 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.728483 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:28.728664 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:28.728732 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:28.728866 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:28.728959 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:28.729023 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.729128 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.729147 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:07:28.731338 27877 raft_consensus.cc:1261] T a1f1eecd93eb4a9dbfdb594726b08635 P f24ddc26670c452ca4569399079eda92 [term 1 FOLLOWER]: Refusing update from remote peer 78b532ff1c354153b2937701392e09cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:28.731338 28166 raft_consensus.cc:1261] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 1 FOLLOWER]: Refusing update from remote peer 78b532ff1c354153b2937701392e09cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:28.731701 28241 consensus_queue.cc:1035] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24ddc26670c452ca4569399079eda92" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:28.731856 28245 consensus_queue.cc:1035] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3df9526891004ee8ab22bc279791c110" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:28.734750 28248 mvcc.cc:205] Tried to move back new op lower bound from 7317269908400410624 to 7317269908364529664. Current Snapshot: MvccSnapshot[applied={T|T < 7317269908400410624}]
I20260811 10:07:28.735162 28249 mvcc.cc:205] Tried to move back new op lower bound from 7317269908400410624 to 7317269908364529664. Current Snapshot: MvccSnapshot[applied={T|T < 7317269908400410624}]
I20260811 10:07:28.737043 27711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:28.737154 27711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:28.737967 27711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d481edc736543fba26f4774f6468078: Sending DeleteTablet for 3 replicas of tablet a1f1eecd93eb4a9dbfdb594726b08635
I20260811 10:07:28.738317 28000 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f1eecd93eb4a9dbfdb594726b08635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:28 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:51382
I20260811 10:07:28.738387 27855 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f1eecd93eb4a9dbfdb594726b08635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:28 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:49312
I20260811 10:07:28.738682 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27785
I20260811 10:07:28.738693 28144 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f1eecd93eb4a9dbfdb594726b08635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:28 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:53720
I20260811 10:07:28.738930 28260 tablet_replica.cc:330] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: stopping tablet replica
I20260811 10:07:28.739032 28260 raft_consensus.cc:2229] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:28.739398 28263 tablet_replica.cc:330] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: stopping tablet replica
I20260811 10:07:28.739503 28263 raft_consensus.cc:2229] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:28.739609 28263 raft_consensus.cc:2258] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:28.739737 28260 raft_consensus.cc:2258] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:28.740995 28263 ts_tablet_manager.cc:1905] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:28.743357 28260 ts_tablet_manager.cc:1905] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:28.745110 28263 ts_tablet_manager.cc:1918] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:28.745201 28263 log.cc:1196] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/a1f1eecd93eb4a9dbfdb594726b08635
I20260811 10:07:28.745349 28260 ts_tablet_manager.cc:1918] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:28.745492 28260 log.cc:1196] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/a1f1eecd93eb4a9dbfdb594726b08635
I20260811 10:07:28.745507 28263 ts_tablet_manager.cc:1939] T a1f1eecd93eb4a9dbfdb594726b08635 P 3df9526891004ee8ab22bc279791c110: Deleting consensus metadata
I20260811 10:07:28.746001 28260 ts_tablet_manager.cc:1939] T a1f1eecd93eb4a9dbfdb594726b08635 P 78b532ff1c354153b2937701392e09cb: Deleting consensus metadata
I20260811 10:07:28.746099 27694 catalog_manager.cc:4841] TS 3df9526891004ee8ab22bc279791c110 (127.22.177.3:36627): tablet a1f1eecd93eb4a9dbfdb594726b08635 (table test-table [id=58a0ad05d2944f6ca5330a8378719ca1]) successfully deleted
I20260811 10:07:28.746683 27695 catalog_manager.cc:4841] TS 78b532ff1c354153b2937701392e09cb (127.22.177.2:40069): tablet a1f1eecd93eb4a9dbfdb594726b08635 (table test-table [id=58a0ad05d2944f6ca5330a8378719ca1]) successfully deleted
W20260811 10:07:28.749332 27692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:36781 (error 108)
I20260811 10:07:28.749572 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27930
I20260811 10:07:28.751298 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.751122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36781 (local address 127.0.0.1:49324)
0811 10:07:28.751192 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:28.751247 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:36781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 10:07:28.751467 27692 catalog_manager.cc:4584] TS f24ddc26670c452ca4569399079eda92 (127.22.177.1:36781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1f1eecd93eb4a9dbfdb594726b08635: Network error: Client connection negotiation failed: client connection to 127.22.177.1:36781: connect: Connection refused (error 111)
I20260811 10:07:28.757017 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28075
I20260811 10:07:28.763842 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 27676
2026-08-11T10:07:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2134 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T10:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:28Z Disabled control of system clock
I20260811 10:07:28.798831 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45719
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:45719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 10:07:28.886157 28268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:28.886392 28268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:28.886416 28268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:28.887588 28268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:28.887626 28268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:28.887639 28268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:28.887651 28268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:28.887663 28268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:28.888993 28268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:45719
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:28.889204 28268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:28.889477 28268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:28.894990 28278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.895046 28276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:28.895411 28275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:28.895493 28268 server_base.cc:1034] running on GCE node
I20260811 10:07:28.895944 28268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:28.896386 28268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:28.897568 28268 hybrid_clock.cc:648] HybridClock initialized: now 1786442848897550 us; error 48 us; skew 500 ppm
I20260811 10:07:28.899207 28268 webserver.cc:460] Webserver started at http://127.22.177.62:42417/ using document root <none> and password file <none>
I20260811 10:07:28.899585 28268 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:28.899670 28268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:28.899854 28268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:28.901074 28268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "da80df3df4764010ac6a6be31fb8947e"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "9309079cc6b0f9cfd5c593e0993538be"
server_key_iv: "bff146f7b485b57416af3c1bc17adad1"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.901500 28268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "da80df3df4764010ac6a6be31fb8947e"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "9309079cc6b0f9cfd5c593e0993538be"
server_key_iv: "bff146f7b485b57416af3c1bc17adad1"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.903090 28268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:28.904171 28284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:28.904451 28268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:28.904569 28268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "da80df3df4764010ac6a6be31fb8947e"
format_stamp: "Formatted at 2026-08-11 10:07:28 on dist-test-slave-g350"
server_key: "9309079cc6b0f9cfd5c593e0993538be"
server_key_iv: "bff146f7b485b57416af3c1bc17adad1"
server_key_version: "encryptionkey@0"
I20260811 10:07:28.904650 28268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:28.908303 28268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:28.908597 28268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:28.908741 28268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:28.912839 28268 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:45719
I20260811 10:07:28.912873 28339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:45719 every 8 connection(s)
I20260811 10:07:28.913225 28268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:28.913980 28340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:28.914208 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28268
I20260811 10:07:28.914309 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:28.914523 23236 external_mini_cluster.cc:1428] Setting key b9232db6ec9ad3e5ffefb9cab31f1294
I20260811 10:07:28.917575 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:28.915169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.22.177.62:45719)
0811 10:07:28.915395 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:28.915404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:28.915439 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:07:28.916024 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:28.916033 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:28.916090 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:28.916265 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:28.917079 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:28.917083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:28.917083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:28.917088 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:28.917149 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:28.917173 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:28.917253 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:28.917306 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 10:07:28.917362 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260811 10:07:28.918429 28340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: Bootstrap starting.
I20260811 10:07:28.919070 28340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:28.919323 28340 log.cc:824] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:28.920001 28340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: No bootstrap required, opened a new log
I20260811 10:07:28.921407 28340 raft_consensus.cc:348] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da80df3df4764010ac6a6be31fb8947e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45719 } }
I20260811 10:07:28.921551 28340 raft_consensus.cc:374] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:28.921574 28340 raft_consensus.cc:729] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da80df3df4764010ac6a6be31fb8947e, State: Initialized, Role: FOLLOWER
I20260811 10:07:28.921809 28340 consensus_queue.cc:260] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da80df3df4764010ac6a6be31fb8947e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45719 } }
I20260811 10:07:28.921950 28340 raft_consensus.cc:388] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:28.921991 28340 raft_consensus.cc:482] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:28.922043 28340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:28.922614 28340 raft_consensus.cc:504] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da80df3df4764010ac6a6be31fb8947e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45719 } }
I20260811 10:07:28.922735 28340 leader_election.cc:302] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da80df3df4764010ac6a6be31fb8947e; no voters:
I20260811 10:07:28.922930 28340 leader_election.cc:288] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:28.923184 28340 sys_catalog.cc:564] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:28.923372 28347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:28.923547 28347 raft_consensus.cc:686] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [term 1 LEADER]: Becoming Leader. State: Replica: da80df3df4764010ac6a6be31fb8947e, State: Running, Role: LEADER
I20260811 10:07:28.923676 28347 consensus_queue.cc:237] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da80df3df4764010ac6a6be31fb8947e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45719 } }
I20260811 10:07:28.924196 28349 sys_catalog.cc:455] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [sys.catalog]: SysCatalogTable state changed. Reason: New leader da80df3df4764010ac6a6be31fb8947e. Latest consensus state: current_term: 1 leader_uuid: "da80df3df4764010ac6a6be31fb8947e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da80df3df4764010ac6a6be31fb8947e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45719 } } }
I20260811 10:07:28.924299 28349 sys_catalog.cc:458] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:28.924391 28348 sys_catalog.cc:455] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da80df3df4764010ac6a6be31fb8947e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da80df3df4764010ac6a6be31fb8947e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45719 } } }
I20260811 10:07:28.924481 28348 sys_catalog.cc:458] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e [sys.catalog]: This master's current role is: LEADER
W20260811 10:07:28.925361 28364 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:28.925433 28364 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:28.925493 28367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:28.925670 28367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:28.927259 28367 catalog_manager.cc:1347] Generated new cluster ID: fdac8fa905db4bad88c0061636dff6f0
I20260811 10:07:28.927327 28367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:28.941501 28367 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:28.941741 28367 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:28.945833 28367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: Generated new TSK 0
I20260811 10:07:28.946043 28367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:07:28.956686 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45719
--builtin_ntp_servers=127.22.177.20:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 10:07:29.056807 28370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:29.057020 28370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:29.057044 28370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:29.058475 28370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:29.058524 28370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:29.058553 28370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:29.059962 28370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:29.060245 28370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:29.060582 28370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:29.066941 28378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.066955 28375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.066955 28376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:29.067448 28370 server_base.cc:1034] running on GCE node
I20260811 10:07:29.067662 28370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:29.067925 28370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:29.069105 28370 hybrid_clock.cc:648] HybridClock initialized: now 1786442849069089 us; error 30 us; skew 500 ppm
I20260811 10:07:29.070698 28370 webserver.cc:460] Webserver started at http://127.22.177.1:41599/ using document root <none> and password file <none>
I20260811 10:07:29.070990 28370 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:29.071062 28370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:29.071238 28370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:29.072352 28370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "1c7521429fc549569c41ba7df5a7335a"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "93ba3abb1deccc4ebc383f2478394790"
server_key_iv: "4dd71069b36665813c6cc422e1f85eb8"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.072781 28370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "1c7521429fc549569c41ba7df5a7335a"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "93ba3abb1deccc4ebc383f2478394790"
server_key_iv: "4dd71069b36665813c6cc422e1f85eb8"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.074390 28370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260811 10:07:29.075263 28385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.075466 28370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:29.075563 28370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "1c7521429fc549569c41ba7df5a7335a"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "93ba3abb1deccc4ebc383f2478394790"
server_key_iv: "4dd71069b36665813c6cc422e1f85eb8"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.075644 28370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:29.079279 28370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:29.079587 28370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:29.079736 28370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:29.079981 28370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:29.080333 28370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:29.080366 28370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.080389 28370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:29.080401 28370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.087477 28370 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:37961
I20260811 10:07:29.087535 28503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:37961 every 8 connection(s)
I20260811 10:07:29.087913 28370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:29.091428 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.088753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:60931 (local address 127.22.177.62:45719)
0811 10:07:29.088847 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.088849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.089147 (+ 298us) server_negotiation.cc:407] Connection header received
0811 10:07:29.089819 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.089821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.089826 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.089877 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.091081 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.091082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.091082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.091087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.091106 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.091111 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.091186 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.091351 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.091366 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 10:07:29.091662 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.088727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45719 (local address 127.22.177.1:60931)
0811 10:07:29.089004 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.089044 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.089701 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.089941 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.089945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.089954 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.090903 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.090904 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.090935 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.090966 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.091187 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.091190 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.091192 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.091321 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.091378 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":122,"threads_started":1}
I20260811 10:07:29.092461 28504 heartbeater.cc:347] Connected to a master server at 127.22.177.62:45719
I20260811 10:07:29.092589 28504 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:29.092796 28504 heartbeater.cc:511] Master 127.22.177.62:45719 requested a full tablet report, sending...
I20260811 10:07:29.093503 28302 ts_manager.cc:194] Registered new tserver with Master: 1c7521429fc549569c41ba7df5a7335a (127.22.177.1:37961)
I20260811 10:07:29.093840 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28370
I20260811 10:07:29.093969 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:29.094194 23236 external_mini_cluster.cc:1428] Setting key b990109137c6e6649612150e52136dba
I20260811 10:07:29.095171 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45719
--builtin_ntp_servers=127.22.177.20:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 10:07:29.185115 28509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:29.185365 28509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:29.185403 28509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:29.186611 28509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:29.186682 28509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:29.186728 28509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:29.188115 28509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:29.188395 28509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:29.188705 28509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:29.194487 28517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.194487 28515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.194481 28514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:29.194782 28509 server_base.cc:1034] running on GCE node
I20260811 10:07:29.195034 28509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:29.195406 28509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:29.196627 28509 hybrid_clock.cc:648] HybridClock initialized: now 1786442849196615 us; error 53 us; skew 500 ppm
I20260811 10:07:29.201720 28509 webserver.cc:460] Webserver started at http://127.22.177.2:46413/ using document root <none> and password file <none>
I20260811 10:07:29.202023 28509 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:29.202099 28509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:29.202345 28509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:29.203485 28509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "a6827bd1d0284a92943d404283a044f8"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "218b3eaf394964c8c05a04a5c424bcbf"
server_key_iv: "6ca0a6a3f44b905856de2165f166b49e"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.203861 28509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "a6827bd1d0284a92943d404283a044f8"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "218b3eaf394964c8c05a04a5c424bcbf"
server_key_iv: "6ca0a6a3f44b905856de2165f166b49e"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.205303 28509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:29.206374 28523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.206584 28509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:29.206738 28509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "a6827bd1d0284a92943d404283a044f8"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "218b3eaf394964c8c05a04a5c424bcbf"
server_key_iv: "6ca0a6a3f44b905856de2165f166b49e"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.206830 28509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:29.210912 28509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:29.211295 28509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:29.211475 28509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:29.211733 28509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:29.212090 28509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:29.212150 28509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.212189 28509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:29.212224 28509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.219285 28509 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:45527
I20260811 10:07:29.219354 28644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:45527 every 8 connection(s)
I20260811 10:07:29.219763 28509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:29.221824 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28509
I20260811 10:07:29.221957 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:29.222193 23236 external_mini_cluster.cc:1428] Setting key 0ba1148513634ee2ea702e8fee0e9695
I20260811 10:07:29.223366 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45719
--builtin_ntp_servers=127.22.177.20:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 10:07:29.223373 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.220681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:42135 (local address 127.22.177.62:45719)
0811 10:07:29.220748 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.220751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.220995 (+ 244us) server_negotiation.cc:407] Connection header received
0811 10:07:29.221620 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.221621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.221629 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.221693 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.223109 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.223110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.223110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.223116 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.223131 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.223135 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.223187 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.223286 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.223302 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:07:29.223732 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.220695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45719 (local address 127.22.177.2:42135)
0811 10:07:29.220872 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.220908 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.221454 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.221789 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.221792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.221800 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.222964 (+ 1164us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.222964 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.222994 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.223022 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.223184 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.223188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.223192 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.223294 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.223358 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260811 10:07:29.224152 28645 heartbeater.cc:347] Connected to a master server at 127.22.177.62:45719
I20260811 10:07:29.224267 28645 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:29.224417 28645 heartbeater.cc:511] Master 127.22.177.62:45719 requested a full tablet report, sending...
I20260811 10:07:29.224754 28302 ts_manager.cc:194] Registered new tserver with Master: a6827bd1d0284a92943d404283a044f8 (127.22.177.2:45527)
W20260811 10:07:29.312074 28649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:29.312300 28649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:29.312323 28649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:29.313521 28649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:29.313562 28649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:29.313588 28649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:29.315083 28649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:45719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:29.315346 28649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:29.315651 28649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:29.321180 28655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.321206 28658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.321266 28656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:29.321580 28649 server_base.cc:1034] running on GCE node
I20260811 10:07:29.321898 28649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:29.322165 28649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:29.323339 28649 hybrid_clock.cc:648] HybridClock initialized: now 1786442849323330 us; error 35 us; skew 500 ppm
I20260811 10:07:29.324898 28649 webserver.cc:460] Webserver started at http://127.22.177.3:36565/ using document root <none> and password file <none>
I20260811 10:07:29.325163 28649 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:29.325237 28649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:29.325395 28649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:29.326566 28649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "7b87d182c1ea4e70992ecd289ee197aa"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "c64e52e1ad2f6eb32879c35f179d96b5"
server_key_iv: "3d74d6f07f8501f423d247408bb57aa1"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.326943 28649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "7b87d182c1ea4e70992ecd289ee197aa"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "c64e52e1ad2f6eb32879c35f179d96b5"
server_key_iv: "3d74d6f07f8501f423d247408bb57aa1"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.328305 28649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:07:29.329063 28665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.329247 28649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:29.329392 28649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "7b87d182c1ea4e70992ecd289ee197aa"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "c64e52e1ad2f6eb32879c35f179d96b5"
server_key_iv: "3d74d6f07f8501f423d247408bb57aa1"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.329485 28649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:29.332855 28649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:29.333124 28649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:29.333272 28649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:29.333578 28649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:29.333992 28649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:29.334057 28649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.334100 28649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:29.334127 28649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.340214 28649 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:33023
I20260811 10:07:29.340260 28789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:33023 every 8 connection(s)
I20260811 10:07:29.340595 28649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:29.343833 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.341416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:35215 (local address 127.22.177.62:45719)
0811 10:07:29.341517 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.341520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.341774 (+ 254us) server_negotiation.cc:407] Connection header received
0811 10:07:29.342242 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.342244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.342251 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.342313 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.343551 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.343552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.343552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.343556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.343572 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.343576 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.343635 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.343758 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.343779 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 10:07:29.343956 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.341386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45719 (local address 127.22.177.3:35215)
0811 10:07:29.341632 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.341677 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.342128 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.342370 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.342374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.342382 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.343385 (+ 1003us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.343385 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.343415 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.343444 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.343648 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.343652 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.343655 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.343716 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.343766 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":120,"threads_started":1}
I20260811 10:07:29.344516 28790 heartbeater.cc:347] Connected to a master server at 127.22.177.62:45719
I20260811 10:07:29.344642 28790 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:29.344827 28790 heartbeater.cc:511] Master 127.22.177.62:45719 requested a full tablet report, sending...
I20260811 10:07:29.345278 28302 ts_manager.cc:194] Registered new tserver with Master: 7b87d182c1ea4e70992ecd289ee197aa (127.22.177.3:33023)
I20260811 10:07:29.350046 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28649
I20260811 10:07:29.350138 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:29.350299 23236 external_mini_cluster.cc:1428] Setting key ec6478cb870544990253e9753db7bc9f
I20260811 10:07:29.351162 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:07:29.353842 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.352650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.22.177.62:45719)
0811 10:07:29.352714 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.352716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.352837 (+ 121us) server_negotiation.cc:407] Connection header received
0811 10:07:29.352902 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.352903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.352908 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.352954 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.353532 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.353533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.353533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.353537 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.353551 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.353555 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.353638 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.353726 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.353737 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:07:29.355727 28302 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:29.357386 28302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:29.361797 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.359805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:37961 (local address 127.0.0.1:40334)
0811 10:07:29.360005 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.360050 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.360104 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.360704 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.360711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.360717 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.361281 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.361282 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.361300 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.361327 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.361585 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.361616 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.361620 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.361705 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.361722 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260811 10:07:29.361826 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.360023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.22.177.1:37961)
0811 10:07:29.360216 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.360220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.360233 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:07:29.360262 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.360264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.360275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.360355 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.361405 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.361438 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.361438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.361444 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.361480 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.361505 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.361576 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.361723 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.361743 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260811 10:07:29.361989 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.360018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:45527 (local address 127.0.0.1:46620)
0811 10:07:29.360282 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.360293 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.360337 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.360754 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.360755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.360758 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.361563 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.361564 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.361571 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.361584 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.361841 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.361842 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.361843 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.361888 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.361902 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":93,"threads_started":1}
I20260811 10:07:29.362134 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.359933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.22.177.3:33023)
0811 10:07:29.360134 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.360139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.360338 (+ 199us) server_negotiation.cc:407] Connection header received
0811 10:07:29.360433 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.360440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.360451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.360526 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.361807 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.361814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.361814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.361819 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.361853 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.361881 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.361919 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.362036 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.362053 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260811 10:07:29.362149 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.360234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.22.177.2:45527)
0811 10:07:29.360578 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.360583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.360592 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:29.360617 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.360619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.360628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.360963 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.361676 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.361684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.361684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.361689 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.361731 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.361757 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.361814 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.362056 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.362072 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":94,"threads_started":1}
I20260811 10:07:29.362305 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.359815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:33023 (local address 127.0.0.1:37816)
0811 10:07:29.360263 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.360273 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.360363 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.360825 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.360826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.360828 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.361700 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.361701 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.361706 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.361715 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.361964 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.361964 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.361965 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.362227 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.362241 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":128,"threads_started":1}
I20260811 10:07:29.362910 28434 tablet_service.cc:1467] Processing CreateTablet for tablet 07ebd7b16d9e42ada0c4949b3f740b0e (DEFAULT_TABLE table=test-table [id=aa22641d2b874025acdb3ded1f606702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:29.362906 28575 tablet_service.cc:1467] Processing CreateTablet for tablet 07ebd7b16d9e42ada0c4949b3f740b0e (DEFAULT_TABLE table=test-table [id=aa22641d2b874025acdb3ded1f606702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:29.363161 28718 tablet_service.cc:1467] Processing CreateTablet for tablet 07ebd7b16d9e42ada0c4949b3f740b0e (DEFAULT_TABLE table=test-table [id=aa22641d2b874025acdb3ded1f606702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:29.363299 28434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ebd7b16d9e42ada0c4949b3f740b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:29.363299 28575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ebd7b16d9e42ada0c4949b3f740b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:29.363533 28718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ebd7b16d9e42ada0c4949b3f740b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:29.366144 28805 tablet_bootstrap.cc:492] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Bootstrap starting.
I20260811 10:07:29.366650 28807 tablet_bootstrap.cc:492] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Bootstrap starting.
I20260811 10:07:29.367025 28805 tablet_bootstrap.cc:654] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:29.367326 28807 tablet_bootstrap.cc:654] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:29.367388 28805 log.cc:824] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:29.367676 28807 log.cc:824] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:29.368036 28806 tablet_bootstrap.cc:492] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a: Bootstrap starting.
I20260811 10:07:29.368357 28805 tablet_bootstrap.cc:492] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: No bootstrap required, opened a new log
I20260811 10:07:29.368430 28805 ts_tablet_manager.cc:1397] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:29.368737 28806 tablet_bootstrap.cc:654] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:29.368846 28807 tablet_bootstrap.cc:492] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: No bootstrap required, opened a new log
I20260811 10:07:29.368917 28807 ts_tablet_manager.cc:1397] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:29.369069 28806 log.cc:824] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:29.369827 28806 tablet_bootstrap.cc:492] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a: No bootstrap required, opened a new log
I20260811 10:07:29.369920 28806 ts_tablet_manager.cc:1397] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:29.371212 28805 raft_consensus.cc:348] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.371407 28805 raft_consensus.cc:374] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:29.371445 28805 raft_consensus.cc:729] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b87d182c1ea4e70992ecd289ee197aa, State: Initialized, Role: FOLLOWER
I20260811 10:07:29.371580 28805 consensus_queue.cc:260] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.371573 28807 raft_consensus.cc:348] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.371738 28807 raft_consensus.cc:374] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:29.371785 28807 raft_consensus.cc:729] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6827bd1d0284a92943d404283a044f8, State: Initialized, Role: FOLLOWER
I20260811 10:07:29.371867 28805 ts_tablet_manager.cc:1428] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 10:07:29.371904 28807 consensus_queue.cc:260] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.371999 28790 heartbeater.cc:503] Master 127.22.177.62:45719 was elected leader, sending a full tablet report...
I20260811 10:07:29.372179 28807 ts_tablet_manager.cc:1428] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 10:07:29.372325 28645 heartbeater.cc:503] Master 127.22.177.62:45719 was elected leader, sending a full tablet report...
I20260811 10:07:29.372225 28806 raft_consensus.cc:348] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.372709 28806 raft_consensus.cc:374] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:29.372763 28806 raft_consensus.cc:729] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7521429fc549569c41ba7df5a7335a, State: Initialized, Role: FOLLOWER
I20260811 10:07:29.372881 28806 consensus_queue.cc:260] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.373147 28806 ts_tablet_manager.cc:1428] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:07:29.373282 28504 heartbeater.cc:503] Master 127.22.177.62:45719 was elected leader, sending a full tablet report...
W20260811 10:07:29.470443 28647 tablet.cc:2369] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:29.480577 28811 raft_consensus.cc:482] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:29.480792 28811 raft_consensus.cc:504] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.481179 28811 leader_election.cc:288] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7521429fc549569c41ba7df5a7335a (127.22.177.1:37961), a6827bd1d0284a92943d404283a044f8 (127.22.177.2:45527)
I20260811 10:07:29.482730 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.481501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:37961 (local address 127.22.177.3:55291)
0811 10:07:29.481568 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.481627 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.481683 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.481853 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.481854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.481857 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.482401 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.482401 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.482408 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.482419 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.482603 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.482604 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.482605 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.482650 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.482663 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 10:07:29.482785 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.481633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:55291 (local address 127.22.177.1:37961)
0811 10:07:29.481700 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.481702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.481712 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:29.481770 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.481771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.481777 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.481838 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.482506 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.482507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.482508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.482512 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.482528 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.482532 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.482606 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.482692 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.482707 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:07:29.483090 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.481764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:41723 (local address 127.22.177.2:45527)
0811 10:07:29.481850 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.481852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.482030 (+ 178us) server_negotiation.cc:407] Connection header received
0811 10:07:29.482061 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.482062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.482067 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.482127 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.482680 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.482681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.482682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.482687 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.482707 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.482712 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.482775 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.483028 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.483044 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 10:07:29.483107 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.481761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:45527 (local address 127.22.177.3:41723)
0811 10:07:29.481935 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.481945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.482005 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.482139 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.482140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.482142 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:29.482558 (+ 416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:07:29.482559 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:29.482568 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:07:29.482586 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:29.482773 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:29.482773 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:29.482774 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:07:29.483011 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 10:07:29.483024 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
I20260811 10:07:29.483070 28455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ebd7b16d9e42ada0c4949b3f740b0e" candidate_uuid: "7b87d182c1ea4e70992ecd289ee197aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7521429fc549569c41ba7df5a7335a" is_pre_election: true
I20260811 10:07:29.483237 28455 raft_consensus.cc:2454] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b87d182c1ea4e70992ecd289ee197aa in term 0.
I20260811 10:07:29.483515 28668 leader_election.cc:302] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7521429fc549569c41ba7df5a7335a, 7b87d182c1ea4e70992ecd289ee197aa; no voters:
I20260811 10:07:29.483662 28811 raft_consensus.cc:2781] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:29.483645 28596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ebd7b16d9e42ada0c4949b3f740b0e" candidate_uuid: "7b87d182c1ea4e70992ecd289ee197aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6827bd1d0284a92943d404283a044f8" is_pre_election: true
I20260811 10:07:29.483712 28811 raft_consensus.cc:482] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:29.483744 28811 raft_consensus.cc:3037] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:29.483814 28596 raft_consensus.cc:2454] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b87d182c1ea4e70992ecd289ee197aa in term 0.
I20260811 10:07:29.484524 28811 raft_consensus.cc:504] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.484661 28811 leader_election.cc:288] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [CANDIDATE]: Term 1 election: Requested vote from peers 1c7521429fc549569c41ba7df5a7335a (127.22.177.1:37961), a6827bd1d0284a92943d404283a044f8 (127.22.177.2:45527)
I20260811 10:07:29.484814 28455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ebd7b16d9e42ada0c4949b3f740b0e" candidate_uuid: "7b87d182c1ea4e70992ecd289ee197aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7521429fc549569c41ba7df5a7335a"
I20260811 10:07:29.484840 28596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ebd7b16d9e42ada0c4949b3f740b0e" candidate_uuid: "7b87d182c1ea4e70992ecd289ee197aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6827bd1d0284a92943d404283a044f8"
I20260811 10:07:29.484925 28455 raft_consensus.cc:3037] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:29.484918 28596 raft_consensus.cc:3037] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:29.485694 28596 raft_consensus.cc:2454] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b87d182c1ea4e70992ecd289ee197aa in term 1.
I20260811 10:07:29.485780 28455 raft_consensus.cc:2454] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b87d182c1ea4e70992ecd289ee197aa in term 1.
I20260811 10:07:29.485862 28668 leader_election.cc:302] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b87d182c1ea4e70992ecd289ee197aa, a6827bd1d0284a92943d404283a044f8; no voters:
I20260811 10:07:29.485976 28811 raft_consensus.cc:2781] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:29.486119 28811 raft_consensus.cc:686] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 1 LEADER]: Becoming Leader. State: Replica: 7b87d182c1ea4e70992ecd289ee197aa, State: Running, Role: LEADER
I20260811 10:07:29.486227 28811 consensus_queue.cc:237] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } }
I20260811 10:07:29.487206 28300 catalog_manager.cc:5495] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b87d182c1ea4e70992ecd289ee197aa (127.22.177.3). New cstate: current_term: 1 leader_uuid: "7b87d182c1ea4e70992ecd289ee197aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b87d182c1ea4e70992ecd289ee197aa" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 33023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:29.512944 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.511787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.22.177.3:33023)
0811 10:07:29.511859 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.511861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.511869 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:29.511918 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.511919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.511935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.512006 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.512675 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.512677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.512677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.512682 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.512700 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.512704 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.512782 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.512849 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.512864 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:07:29.515013 28596 raft_consensus.cc:1261] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 1 FOLLOWER]: Refusing update from remote peer 7b87d182c1ea4e70992ecd289ee197aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:29.515079 28455 raft_consensus.cc:1261] T 07ebd7b16d9e42ada0c4949b3f740b0e P 1c7521429fc549569c41ba7df5a7335a [term 1 FOLLOWER]: Refusing update from remote peer 7b87d182c1ea4e70992ecd289ee197aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:29.515419 28811 consensus_queue.cc:1035] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6827bd1d0284a92943d404283a044f8" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:29.515584 28815 consensus_queue.cc:1035] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7521429fc549569c41ba7df5a7335a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 37961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:29.519168 28822 mvcc.cc:205] Tried to move back new op lower bound from 7317269911610658816 to 7317269911496622080. Current Snapshot: MvccSnapshot[applied={T|T < 7317269911610658816}]
I20260811 10:07:29.519730 28820 mvcc.cc:205] Tried to move back new op lower bound from 7317269911610658816 to 7317269911496622080. Current Snapshot: MvccSnapshot[applied={T|T < 7317269911610658816 or (T in {7317269911610658816})}]
I20260811 10:07:29.520690 28300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:29.520805 28300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:29.521795 28300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da80df3df4764010ac6a6be31fb8947e: Sending DeleteTablet for 3 replicas of tablet 07ebd7b16d9e42ada0c4949b3f740b0e
I20260811 10:07:29.522253 28718 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ebd7b16d9e42ada0c4949b3f740b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:29 UTC) from {username='slave'} at 127.0.0.1:37816
I20260811 10:07:29.522262 28575 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ebd7b16d9e42ada0c4949b3f740b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:29 UTC) from {username='slave'} at 127.0.0.1:46620
I20260811 10:07:29.522578 28829 tablet_replica.cc:330] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: stopping tablet replica
I20260811 10:07:29.522573 28434 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ebd7b16d9e42ada0c4949b3f740b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:29 UTC) from {username='slave'} at 127.0.0.1:40334
I20260811 10:07:29.522648 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28370
I20260811 10:07:29.522682 28829 raft_consensus.cc:2229] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:29.523401 28829 raft_consensus.cc:2258] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:29.524566 28829 ts_tablet_manager.cc:1905] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:29.526733 28829 ts_tablet_manager.cc:1918] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:29.526805 28829 log.cc:1196] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/07ebd7b16d9e42ada0c4949b3f740b0e
I20260811 10:07:29.527038 28829 ts_tablet_manager.cc:1939] T 07ebd7b16d9e42ada0c4949b3f740b0e P a6827bd1d0284a92943d404283a044f8: Deleting consensus metadata
I20260811 10:07:29.532650 28287 catalog_manager.cc:4841] TS a6827bd1d0284a92943d404283a044f8 (127.22.177.2:45527): tablet 07ebd7b16d9e42ada0c4949b3f740b0e (table test-table [id=aa22641d2b874025acdb3ded1f606702]) successfully deleted
I20260811 10:07:29.534097 28830 tablet_replica.cc:330] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: stopping tablet replica
I20260811 10:07:29.534241 28830 raft_consensus.cc:2229] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:29.534366 28830 raft_consensus.cc:2258] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:29.534917 28830 ts_tablet_manager.cc:1905] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:29.536801 28830 ts_tablet_manager.cc:1918] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:29.536890 28830 log.cc:1196] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/07ebd7b16d9e42ada0c4949b3f740b0e
I20260811 10:07:29.537168 28830 ts_tablet_manager.cc:1939] T 07ebd7b16d9e42ada0c4949b3f740b0e P 7b87d182c1ea4e70992ecd289ee197aa: Deleting consensus metadata
I20260811 10:07:29.537700 28286 catalog_manager.cc:4841] TS 7b87d182c1ea4e70992ecd289ee197aa (127.22.177.3:33023): tablet 07ebd7b16d9e42ada0c4949b3f740b0e (table test-table [id=aa22641d2b874025acdb3ded1f606702]) successfully deleted
W20260811 10:07:29.538658 28287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:37961 (error 108)
I20260811 10:07:29.539011 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28509
I20260811 10:07:29.539245 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.539096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:37961 (local address 127.0.0.1:40340)
0811 10:07:29.539157 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.539207 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 10:07:29.539363 28287 catalog_manager.cc:4584] TS 1c7521429fc549569c41ba7df5a7335a (127.22.177.1:37961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07ebd7b16d9e42ada0c4949b3f740b0e: Network error: Client connection negotiation failed: client connection to 127.22.177.1:37961: connect: Connection refused (error 111)
I20260811 10:07:29.551582 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28649
I20260811 10:07:29.560115 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28268
2026-08-11T10:07:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (792 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 10:07:29.582901 23236 mini_oidc.cc:152] Starting JWKS server
I20260811 10:07:29.584323 23236 webserver.cc:460] Webserver started at https://127.0.0.1:34255/ using document root <none> and password file <none>
I20260811 10:07:29.584476 23236 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 10:07:29.584681 23236 webserver.cc:460] Webserver started at http://127.0.0.1:45091/ using document root <none> and password file <none>
2026-08-11T10:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:29Z Disabled control of system clock
I20260811 10:07:29.594626 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45799
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:38551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:45799
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34255/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/testchainca.cert with env {}
W20260811 10:07:29.678848 28839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:29.679072 28839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:29.679095 28839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:29.680375 28839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:29.680428 28839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:29.680441 28839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:29.680455 28839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:29.680467 28839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:07:29.680510 28839 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:07:29.680537 28839 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34255/jwks.json
I20260811 10:07:29.682353 28839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:45799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34255/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:29.682657 28839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:29.682907 28839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:29.688237 28847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.688233 28844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.688210 28849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:29.688719 28839 server_base.cc:1034] running on GCE node
I20260811 10:07:29.688971 28839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:29.689352 28839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:29.690526 28839 hybrid_clock.cc:648] HybridClock initialized: now 1786442849690515 us; error 48 us; skew 500 ppm
I20260811 10:07:29.691767 28839 webserver.cc:460] Webserver started at http://127.22.177.62:36759/ using document root <none> and password file <none>
I20260811 10:07:29.692018 28839 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:29.692090 28839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:29.692236 28839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:29.693369 28839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "84fc9fba550448e7abcaa6955f2ebdcb"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "03901a2e6f17156e4481176c29931fc9"
server_key_iv: "8d5b960fa288a3d07c49bf1a7cb81e32"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.693765 28839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "84fc9fba550448e7abcaa6955f2ebdcb"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "03901a2e6f17156e4481176c29931fc9"
server_key_iv: "8d5b960fa288a3d07c49bf1a7cb81e32"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.695087 28839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:29.695886 28855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.696065 28839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:29.696170 28839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "84fc9fba550448e7abcaa6955f2ebdcb"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "03901a2e6f17156e4481176c29931fc9"
server_key_iv: "8d5b960fa288a3d07c49bf1a7cb81e32"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.696249 28839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:29.715934 28839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:29.716305 28839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:29.716461 28839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:29.720777 28839 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:45799
I20260811 10:07:29.720819 28911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:45799 every 8 connection(s)
I20260811 10:07:29.721223 28839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:29.722275 28912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:29.724653 28912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb: Bootstrap starting.
I20260811 10:07:29.725294 28912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:29.725627 28912 log.cc:824] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:29.726363 28912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb: No bootstrap required, opened a new log
I20260811 10:07:29.728020 28912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45799 } }
I20260811 10:07:29.728183 28912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:29.728207 28912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84fc9fba550448e7abcaa6955f2ebdcb, State: Initialized, Role: FOLLOWER
I20260811 10:07:29.728400 28912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45799 } }
I20260811 10:07:29.728466 28912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:29.728488 28912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:29.728542 28912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:29.729123 28912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45799 } }
I20260811 10:07:29.729240 28912 leader_election.cc:302] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84fc9fba550448e7abcaa6955f2ebdcb; no voters:
I20260811 10:07:29.729421 28912 leader_election.cc:288] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:29.729480 28915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:29.729691 28915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [term 1 LEADER]: Becoming Leader. State: Replica: 84fc9fba550448e7abcaa6955f2ebdcb, State: Running, Role: LEADER
I20260811 10:07:29.729749 28912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:29.729764 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28839
I20260811 10:07:29.729816 28915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45799 } }
I20260811 10:07:29.729866 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:29.730063 23236 external_mini_cluster.cc:1428] Setting key 29ba3004453d3f446eab3d4603b935e3
I20260811 10:07:29.730433 28916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45799 } } }
I20260811 10:07:29.730518 28916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:29.730779 28917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84fc9fba550448e7abcaa6955f2ebdcb. Latest consensus state: current_term: 1 leader_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc9fba550448e7abcaa6955f2ebdcb" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45799 } } }
I20260811 10:07:29.730885 28925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:29.730912 28917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:29.731470 28925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:29.733173 28925 catalog_manager.cc:1347] Generated new cluster ID: b790dda12c6c422d83252a6050688d69
I20260811 10:07:29.733230 28925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:29.736681 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.730734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.22.177.62:45799)
0811 10:07:29.730953 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.730964 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.731251 (+ 287us) server_negotiation.cc:407] Connection header received
0811 10:07:29.731423 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.731430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.731503 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.731632 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.732072 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.733198 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.734354 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.734974 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.735096 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:29.735896 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.735900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.735901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.736112 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.736153 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.736167 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.736342 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.736386 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.736463 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260811 10:07:29.737736 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:07:29.738363 28925 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:29.739102 28925 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:29.744800 28925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84fc9fba550448e7abcaa6955f2ebdcb: Generated new TSK 0
I20260811 10:07:29.744982 28925 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 10:07:29.770287 28944 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 10:07:29.770463 28944 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 10:07:29.770677 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.754273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.22.177.62:45799)
0811 10:07:29.754322 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.754325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.754485 (+ 160us) server_negotiation.cc:407] Connection header received
0811 10:07:29.754590 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.754591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.754600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.754661 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:07:29.754853 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.755325 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.756516 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.756693 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.756750 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:29.756993 (+ 243us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:07:29.770394 (+ 13401us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 10:07:29.770599 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53660: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 10:07:29.770750 28927 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53660: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 10:07:29.771225 28943 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:29.754176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:45799 (local address 127.0.0.1:53660)
0811 10:07:29.754403 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:29.754416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:29.754491 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:29.754698 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:29.754699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:29.754701 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:07:29.754781 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:29.754783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.755424 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.755425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:29.756427 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:29.756429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.756788 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.756789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:29.756856 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:29.756880 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:07:29.770666 (+ 13786us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 10:07:29.770812 (+ 146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.177.62:45799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.177.62:45799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 10:07:29.771925 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28839
2026-08-11T10:07:29Z chronyd exiting
I20260811 10:07:29.811208 23236 test_util.cc:175] -----------------------------------------------
I20260811 10:07:29.811319 23236 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786442825609099-23236-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (229 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 10:07:29.812178 23236 mini_oidc.cc:152] Starting JWKS server
I20260811 10:07:29.812901 23236 webserver.cc:460] Webserver started at https://127.0.0.1:45651/ using document root <none> and password file <none>
I20260811 10:07:29.812973 23236 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 10:07:29.813144 23236 webserver.cc:460] Webserver started at http://127.0.0.1:44651/ using document root <none> and password file <none>
2026-08-11T10:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:29Z Disabled control of system clock
I20260811 10:07:29.822427 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:39331
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:35995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:39331
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45651/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/testchainca.cert with env {}
W20260811 10:07:29.907711 28951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:29.907939 28951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:29.907963 28951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:29.909198 28951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:29.909247 28951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:29.909260 28951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:29.909273 28951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:29.909286 28951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:07:29.909324 28951 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:07:29.909354 28951 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45651/jwks.json
I20260811 10:07:29.911001 28951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:39331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:39331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45651/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:29.911281 28951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:29.911515 28951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:29.916900 28956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.916914 28957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:29.916900 28959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:29.917168 28951 server_base.cc:1034] running on GCE node
I20260811 10:07:29.917555 28951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:29.917905 28951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:29.919052 28951 hybrid_clock.cc:648] HybridClock initialized: now 1786442849919028 us; error 52 us; skew 500 ppm
I20260811 10:07:29.920500 28951 webserver.cc:460] Webserver started at http://127.22.177.62:40567/ using document root <none> and password file <none>
I20260811 10:07:29.920745 28951 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:29.920802 28951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:29.920992 28951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:29.922195 28951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "7d0169ee8d2f23cf0570e2f1ee6263a0"
server_key_iv: "ea9af09e86f499e6ce0454355f0ee131"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.922574 28951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "7d0169ee8d2f23cf0570e2f1ee6263a0"
server_key_iv: "ea9af09e86f499e6ce0454355f0ee131"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.923939 28951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:29.924860 28966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:29.925179 28951 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:29.925319 28951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b"
format_stamp: "Formatted at 2026-08-11 10:07:29 on dist-test-slave-g350"
server_key: "7d0169ee8d2f23cf0570e2f1ee6263a0"
server_key_iv: "ea9af09e86f499e6ce0454355f0ee131"
server_key_version: "encryptionkey@0"
I20260811 10:07:29.925410 28951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:29.948307 28951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:29.948671 28951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:29.948845 28951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:29.953537 28951 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:39331
I20260811 10:07:29.953662 29025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:39331 every 8 connection(s)
I20260811 10:07:29.954005 28951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:29.954754 29026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:29.957617 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 28951
I20260811 10:07:29.957748 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:29.957927 23236 external_mini_cluster.cc:1428] Setting key 572b43c4a70509e52f5ac8dbc448498a
I20260811 10:07:29.958014 29026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b: Bootstrap starting.
I20260811 10:07:29.958787 29026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:29.959127 29026 log.cc:824] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:29.959961 29026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b: No bootstrap required, opened a new log
I20260811 10:07:29.962280 29026 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39331 } }
I20260811 10:07:29.962466 29026 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:29.962504 29026 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebb64b3b1bab41f3a38e1cbdf878b26b, State: Initialized, Role: FOLLOWER
I20260811 10:07:29.962638 29026 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39331 } }
I20260811 10:07:29.962719 29026 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:29.962751 29026 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:29.962798 29026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:29.963590 29026 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39331 } }
I20260811 10:07:29.963722 29026 leader_election.cc:302] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebb64b3b1bab41f3a38e1cbdf878b26b; no voters:
I20260811 10:07:29.963946 29026 leader_election.cc:288] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:29.964172 29032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:29.964246 29026 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:29.964650 29032 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [term 1 LEADER]: Becoming Leader. State: Replica: ebb64b3b1bab41f3a38e1cbdf878b26b, State: Running, Role: LEADER
I20260811 10:07:29.964893 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:29.958558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56946 (local address 127.22.177.62:39331)
0811 10:07:29.958792 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:29.958801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:29.958842 (+ 41us) server_negotiation.cc:407] Connection header received
0811 10:07:29.959617 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:29.959626 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:29.959707 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:29.959868 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:29.960099 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.961526 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.962410 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:29.963019 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:29.963148 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:29.963924 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:29.963928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:29.963929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:29.964154 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:29.964201 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:29.964218 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:29.964536 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:29.964572 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 10:07:29.964663 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260811 10:07:29.964852 29032 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39331 } }
I20260811 10:07:29.965713 29032 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebb64b3b1bab41f3a38e1cbdf878b26b. Latest consensus state: current_term: 1 leader_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39331 } } }
I20260811 10:07:29.965724 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb64b3b1bab41f3a38e1cbdf878b26b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39331 } } }
I20260811 10:07:29.965834 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:07:29.965883 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:29.966076 29032 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b [sys.catalog]: This master's current role is: LEADER
W20260811 10:07:29.966636 29048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:29.966761 29048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:29.967037 29047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:29.967267 29047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:29.968667 29047 catalog_manager.cc:1347] Generated new cluster ID: 8618dbd4414243e1b606164d25d0e96d
I20260811 10:07:29.968722 29047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:29.980706 29047 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:29.981426 29047 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:29.995855 29047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebb64b3b1bab41f3a38e1cbdf878b26b: Generated new TSK 0
I20260811 10:07:29.996273 29047 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 10:07:30.023149 29058 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 10:07:30.023252 29058 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 10:07:30.023592 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:30.006297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.22.177.62:39331)
0811 10:07:30.006383 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:30.006385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:30.006803 (+ 418us) server_negotiation.cc:407] Connection header received
0811 10:07:30.006858 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:30.006859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:30.006872 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:30.006956 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:07:30.007192 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.007813 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.009494 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.009835 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.009909 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.010172 (+ 263us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:07:30.023401 (+ 13229us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 10:07:30.023537 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 10:07:30.023639 29029 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 10:07:30.024008 29057 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:30.006212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:39331 (local address 127.0.0.1:56950)
0811 10:07:30.006453 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:30.006468 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:30.006557 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:30.006978 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:30.006979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:30.006981 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:07:30.007054 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:30.007057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.008044 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.008045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:30.009365 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:30.009369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.009938 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.009939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:30.010028 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.010036 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:07:30.023540 (+ 13504us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 10:07:30.023903 (+ 363us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.177.62:39331: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":184,"threads_started":1}
I20260811 10:07:30.024446 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 28951
2026-08-11T10:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (254 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 10:07:30.066354 23236 mini_oidc.cc:152] Starting JWKS server
I20260811 10:07:30.067157 23236 webserver.cc:460] Webserver started at https://127.0.0.1:39497/ using document root <none> and password file <none>
I20260811 10:07:30.067246 23236 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 10:07:30.067430 23236 webserver.cc:460] Webserver started at http://127.0.0.1:38403/ using document root <none> and password file <none>
2026-08-11T10:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:30Z Disabled control of system clock
I20260811 10:07:30.077823 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36463
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:36463
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39497/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 10:07:30.162211 29064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:30.162431 29064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:30.162453 29064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:30.163589 29064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:30.163626 29064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:30.163638 29064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:30.163651 29064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:30.163663 29064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:07:30.163676 29064 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:07:30.163686 29064 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39497/jwks.json
I20260811 10:07:30.165043 29064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:36463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39497/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:30.165267 29064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:30.165521 29064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:30.170740 29070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:30.170821 29069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:30.170740 29073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:30.170958 29064 server_base.cc:1034] running on GCE node
I20260811 10:07:30.171221 29064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:30.171566 29064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:30.172746 29064 hybrid_clock.cc:648] HybridClock initialized: now 1786442850172703 us; error 55 us; skew 500 ppm
I20260811 10:07:30.174121 29064 webserver.cc:460] Webserver started at http://127.22.177.62:38195/ using document root <none> and password file <none>
I20260811 10:07:30.174402 29064 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:30.174479 29064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:30.174634 29064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:30.175932 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "7e5915d8ef234b9dbe04cadb7d05f742"
format_stamp: "Formatted at 2026-08-11 10:07:30 on dist-test-slave-g350"
server_key: "408ee2d76111a127f91fbb1d2c3079e4"
server_key_iv: "9a79c4f80d01c2e627b5c0561ba08251"
server_key_version: "encryptionkey@0"
I20260811 10:07:30.176362 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "7e5915d8ef234b9dbe04cadb7d05f742"
format_stamp: "Formatted at 2026-08-11 10:07:30 on dist-test-slave-g350"
server_key: "408ee2d76111a127f91fbb1d2c3079e4"
server_key_iv: "9a79c4f80d01c2e627b5c0561ba08251"
server_key_version: "encryptionkey@0"
I20260811 10:07:30.178076 29064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:30.179026 29079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:30.179237 29064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:30.179373 29064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "7e5915d8ef234b9dbe04cadb7d05f742"
format_stamp: "Formatted at 2026-08-11 10:07:30 on dist-test-slave-g350"
server_key: "408ee2d76111a127f91fbb1d2c3079e4"
server_key_iv: "9a79c4f80d01c2e627b5c0561ba08251"
server_key_version: "encryptionkey@0"
I20260811 10:07:30.179458 29064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:30.199519 29064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:30.199928 29064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:30.200101 29064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:30.204617 29064 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:36463
I20260811 10:07:30.204684 29138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:36463 every 8 connection(s)
I20260811 10:07:30.205060 29064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:30.205839 29139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:30.208742 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742: Bootstrap starting.
I20260811 10:07:30.209358 29139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:30.209712 29139 log.cc:824] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:30.210532 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742: No bootstrap required, opened a new log
I20260811 10:07:30.212153 29139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36463 } }
I20260811 10:07:30.212304 29139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:30.212327 29139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5915d8ef234b9dbe04cadb7d05f742, State: Initialized, Role: FOLLOWER
I20260811 10:07:30.212477 29139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36463 } }
I20260811 10:07:30.212579 29139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:30.212602 29139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:30.212632 29139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:30.213124 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29064
I20260811 10:07:30.213239 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:30.213186 29139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36463 } }
I20260811 10:07:30.213351 29139 leader_election.cc:302] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e5915d8ef234b9dbe04cadb7d05f742; no voters:
I20260811 10:07:30.213454 23236 external_mini_cluster.cc:1428] Setting key 6aa4c8fd4b3b8b0dd3359137061a53ce
I20260811 10:07:30.213542 29139 leader_election.cc:288] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:30.213583 29142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:30.213758 29142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [term 1 LEADER]: Becoming Leader. State: Replica: 7e5915d8ef234b9dbe04cadb7d05f742, State: Running, Role: LEADER
I20260811 10:07:30.213860 29139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:30.213876 29142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36463 } }
I20260811 10:07:30.214237 29143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36463 } } }
I20260811 10:07:30.214319 29143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:30.214959 29144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e5915d8ef234b9dbe04cadb7d05f742. Latest consensus state: current_term: 1 leader_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5915d8ef234b9dbe04cadb7d05f742" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36463 } } }
I20260811 10:07:30.215088 29144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:30.215397 29149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:30.216091 29149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:30.217741 29149 catalog_manager.cc:1347] Generated new cluster ID: 8965860d2b47435fbd77b731302c236f
I20260811 10:07:30.217799 29149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:30.219839 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:30.214070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43450 (local address 127.22.177.62:36463)
0811 10:07:30.214273 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:30.214282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:30.214466 (+ 184us) server_negotiation.cc:407] Connection header received
0811 10:07:30.214740 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:30.214748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:30.214869 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:30.215011 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:30.215437 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.216521 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.217580 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.218204 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.218348 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.219086 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:30.219090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:30.219090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:30.219304 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:30.219349 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:30.219365 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:30.219441 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:30.219516 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 10:07:30.219602 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260811 10:07:30.220743 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:07:30.225452 29149 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:30.226141 29149 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:30.234762 29149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e5915d8ef234b9dbe04cadb7d05f742: Generated new TSK 0
I20260811 10:07:30.235009 29149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:07:30.249895 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:30.245697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.22.177.62:36463)
0811 10:07:30.245754 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:30.245756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:30.245835 (+ 79us) server_negotiation.cc:407] Connection header received
0811 10:07:30.245914 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:30.245915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:30.245929 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:30.246021 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:07:30.246224 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.247039 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.247919 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.248261 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.248342 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.248565 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:07:30.249660 (+ 1095us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 10:07:30.249812 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43460: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 10:07:30.249971 29146 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43460: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 10:07:30.250023 29170 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:30.245554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36463 (local address 127.0.0.1:43460)
0811 10:07:30.245782 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:30.245792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:30.245846 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:30.246064 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:30.246065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:30.246067 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:07:30.246139 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:30.246141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.247142 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.247143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:30.247824 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:30.247827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.248388 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.248390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:30.248441 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.248446 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:07:30.249785 (+ 1339us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 10:07:30.249900 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.177.62:36463: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":140,"threads_started":1}
I20260811 10:07:30.250406 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29064
E20260811 10:07:30.265285 29171 webserver.cc:533] Webserver: sslize failed:
E20260811 10:07:30.265373 29171 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T10:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T10:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:30Z Disabled control of system clock
I20260811 10:07:30.302243 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36055
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:33113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:36055
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/testchainca.cert with env {}
W20260811 10:07:30.386583 29175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:30.386792 29175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:30.386814 29175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:30.387935 29175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:30.387976 29175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:30.387989 29175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:30.388001 29175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:30.388013 29175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:07:30.388026 29175 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:07:30.388037 29175 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 10:07:30.389500 29175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:36055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:30.389726 29175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:30.389887 29175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:30.395105 29181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:30.395120 29185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:30.395190 29175 server_base.cc:1034] running on GCE node
W20260811 10:07:30.395105 29182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:30.395627 29175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:30.395946 29175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:30.397125 29175 hybrid_clock.cc:648] HybridClock initialized: now 1786442850397118 us; error 40 us; skew 500 ppm
I20260811 10:07:30.398605 29175 webserver.cc:460] Webserver started at http://127.22.177.62:40985/ using document root <none> and password file <none>
I20260811 10:07:30.398901 29175 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:30.398973 29175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:30.399129 29175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:30.400341 29175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "ceca3d4a2d85452bbd341732ea40dbb5"
format_stamp: "Formatted at 2026-08-11 10:07:30 on dist-test-slave-g350"
server_key: "b38ea1eec647da8427155ffca13f796d"
server_key_iv: "f2479145695bc8bb0cf32af5a00be19a"
server_key_version: "encryptionkey@0"
I20260811 10:07:30.400766 29175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "ceca3d4a2d85452bbd341732ea40dbb5"
format_stamp: "Formatted at 2026-08-11 10:07:30 on dist-test-slave-g350"
server_key: "b38ea1eec647da8427155ffca13f796d"
server_key_iv: "f2479145695bc8bb0cf32af5a00be19a"
server_key_version: "encryptionkey@0"
I20260811 10:07:30.402335 29175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:30.403172 29192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:30.403410 29175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:30.403529 29175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ceca3d4a2d85452bbd341732ea40dbb5"
format_stamp: "Formatted at 2026-08-11 10:07:30 on dist-test-slave-g350"
server_key: "b38ea1eec647da8427155ffca13f796d"
server_key_iv: "f2479145695bc8bb0cf32af5a00be19a"
server_key_version: "encryptionkey@0"
I20260811 10:07:30.403610 29175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:30.414588 29175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:30.414939 29175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:30.415088 29175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:30.419379 29175 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:36055
I20260811 10:07:30.419413 29247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:36055 every 8 connection(s)
I20260811 10:07:30.419772 29175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:30.420805 29248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:30.423234 29248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5: Bootstrap starting.
I20260811 10:07:30.423858 29248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:30.424116 29248 log.cc:824] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:30.424809 29248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5: No bootstrap required, opened a new log
I20260811 10:07:30.426637 29248 raft_consensus.cc:348] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36055 } }
I20260811 10:07:30.426784 29248 raft_consensus.cc:374] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:30.426810 29248 raft_consensus.cc:729] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceca3d4a2d85452bbd341732ea40dbb5, State: Initialized, Role: FOLLOWER
I20260811 10:07:30.426918 29248 consensus_queue.cc:260] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36055 } }
I20260811 10:07:30.426975 29248 raft_consensus.cc:388] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:30.426996 29248 raft_consensus.cc:482] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:30.427069 29248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:30.427347 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29175
I20260811 10:07:30.427479 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:30.427652 23236 external_mini_cluster.cc:1428] Setting key 99a48bc4ec6df0ae0d3f75d68b155347
I20260811 10:07:30.427729 29248 raft_consensus.cc:504] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36055 } }
I20260811 10:07:30.427850 29248 leader_election.cc:302] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ceca3d4a2d85452bbd341732ea40dbb5; no voters:
I20260811 10:07:30.428045 29248 leader_election.cc:288] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:30.428375 29251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:30.428503 29251 raft_consensus.cc:686] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [term 1 LEADER]: Becoming Leader. State: Replica: ceca3d4a2d85452bbd341732ea40dbb5, State: Running, Role: LEADER
I20260811 10:07:30.428597 29248 sys_catalog.cc:564] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:30.428622 29251 consensus_queue.cc:237] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36055 } }
I20260811 10:07:30.429142 29254 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36055 } } }
I20260811 10:07:30.429226 29254 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:30.429633 29255 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ceca3d4a2d85452bbd341732ea40dbb5. Latest consensus state: current_term: 1 leader_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceca3d4a2d85452bbd341732ea40dbb5" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36055 } } }
I20260811 10:07:30.429708 29255 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:30.430120 29263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:30.430673 29263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:30.432278 29263 catalog_manager.cc:1347] Generated new cluster ID: 0ebd27c00a3d4690992d7e55fa0cd51b
I20260811 10:07:30.432341 29263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:30.433959 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:30.428212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.22.177.62:36055)
0811 10:07:30.428425 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:30.428455 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:30.428489 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:07:30.428611 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:30.428617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:30.428695 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:30.429092 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:30.429194 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.430659 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.431726 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.432273 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.432415 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.433240 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:30.433244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:30.433244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:07:30.433427 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:30.433458 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:07:30.433468 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:30.433544 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:30.433655 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 10:07:30.433736 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260811 10:07:30.434844 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:07:30.455283 29263 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:30.456027 29263 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:30.463583 29263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ceca3d4a2d85452bbd341732ea40dbb5: Generated new TSK 0
I20260811 10:07:30.463788 29263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:07:30.562192 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:30.476441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.22.177.62:36055)
0811 10:07:30.476499 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:30.476501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:30.476570 (+ 69us) server_negotiation.cc:407] Connection header received
0811 10:07:30.476617 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:30.476618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:30.476631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:30.476731 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:07:30.476897 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.477743 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.478513 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.478819 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.478904 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.479061 (+ 157us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:07:30.561830 (+ 82769us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 10:07:30.562082 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34402: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 10:07:30.562255 29278 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:30.476315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36055 (local address 127.0.0.1:34402)
0811 10:07:30.476488 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:30.476499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:30.476561 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:30.476735 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:30.476736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:30.476738 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:07:30.476826 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:30.476828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.477828 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.477829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:30.478422 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:30.478424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:30.478911 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:30.478911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:30.478951 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:30.478956 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:07:30.562027 (+ 83071us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 10:07:30.562162 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.177.62:36055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":119,"threads_started":1}
W20260811 10:07:30.562292 29253 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34402: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260811 10:07:30.562631 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29175
2026-08-11T10:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (291 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:30 dist-test-slave-g350 krb5kdc[29293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:30 dist-test-slave-g350 krb5kdc[29293](info): set up 2 sockets
Aug 11 10:07:30 dist-test-slave-g350 krb5kdc[29293](info): commencing operation
krb5kdc: starting...
W20260811 10:07:31.894582 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:31 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:32.035058 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37467
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:32.128721 29307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:32.129027 29307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:32.129076 29307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:32.130515 29307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:32.130605 29307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:32.130635 29307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:32.130669 29307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:32.130695 29307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:32.132411 29307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:32.132733 29307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:32.132998 29307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:32.138934 29307 server_base.cc:1034] running on GCE node
W20260811 10:07:32.138917 29317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.139039 29315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.139021 29314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:32.139520 29307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:32.139880 29307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:32.141075 29307 hybrid_clock.cc:648] HybridClock initialized: now 1786442852141054 us; error 47 us; skew 500 ppm
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.143872 29307 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:32.144454 29307 webserver.cc:460] Webserver started at http://127.22.177.62:39885/ using document root <none> and password file <none>
I20260811 10:07:32.144696 29307 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:32.144774 29307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:32.144937 29307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:32.146162 29307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "b7f72d21fa194e518e43149647b62182"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "26c8104a3b3a3ddda3a0ff21fa4c0ca9"
server_key_iv: "1c11383dc94fff0fc5c2ebf58577e291"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.146555 29307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "b7f72d21fa194e518e43149647b62182"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "26c8104a3b3a3ddda3a0ff21fa4c0ca9"
server_key_iv: "1c11383dc94fff0fc5c2ebf58577e291"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.148083 29307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:32.149065 29324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.149359 29307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:32.149480 29307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "b7f72d21fa194e518e43149647b62182"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "26c8104a3b3a3ddda3a0ff21fa4c0ca9"
server_key_iv: "1c11383dc94fff0fc5c2ebf58577e291"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.149559 29307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:32.160508 29307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:32.163089 29307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:32.163313 29307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:32.167716 29307 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37467
I20260811 10:07:32.167781 29381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37467 every 8 connection(s)
I20260811 10:07:32.168197 29307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:32.169201 29382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:32.171459 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29307
I20260811 10:07:32.171564 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:32.171676 29382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182: Bootstrap starting.
I20260811 10:07:32.171835 23236 external_mini_cluster.cc:1428] Setting key 0ce23a60111017f7898ad50bd0662683
I20260811 10:07:32.172385 29382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:32.172720 29382 log.cc:824] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:32.173444 29382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182: No bootstrap required, opened a new log
I20260811 10:07:32.175354 29382 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f72d21fa194e518e43149647b62182" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37467 } }
I20260811 10:07:32.175514 29382 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:32.175563 29382 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f72d21fa194e518e43149647b62182, State: Initialized, Role: FOLLOWER
I20260811 10:07:32.175697 29382 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f72d21fa194e518e43149647b62182" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37467 } }
I20260811 10:07:32.175804 29382 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:32.175849 29382 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:32.175910 29382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:32.176600 29382 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f72d21fa194e518e43149647b62182" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37467 } }
I20260811 10:07:32.176725 29382 leader_election.cc:302] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7f72d21fa194e518e43149647b62182; no voters:
I20260811 10:07:32.176901 29382 leader_election.cc:288] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:32.176923 29388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:32.177054 29388 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [term 1 LEADER]: Becoming Leader. State: Replica: b7f72d21fa194e518e43149647b62182, State: Running, Role: LEADER
I20260811 10:07:32.177140 29388 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f72d21fa194e518e43149647b62182" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37467 } }
I20260811 10:07:32.177175 29382 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:32.177470 29388 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7f72d21fa194e518e43149647b62182. Latest consensus state: current_term: 1 leader_uuid: "b7f72d21fa194e518e43149647b62182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f72d21fa194e518e43149647b62182" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37467 } } }
I20260811 10:07:32.177949 29388 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:32.177486 29389 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7f72d21fa194e518e43149647b62182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f72d21fa194e518e43149647b62182" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37467 } } }
I20260811 10:07:32.178105 29389 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182 [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:32.178501 29396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:32.179059 29396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:32.180863 29396 catalog_manager.cc:1347] Generated new cluster ID: 36ecc630b2fa450c8a46846f561c8de2
I20260811 10:07:32.180930 29396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:32.184289 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.172393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.22.177.62:37467)
0811 10:07:32.172606 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.172615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.172648 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:07:32.173125 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.173132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.173180 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.173408 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.174107 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.175110 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.176010 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.176637 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.176766 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.180132 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.180136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.180137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.180149 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.182512 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.182961 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.182964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.182965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.183079 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.183327 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.183327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.183328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.183634 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.183810 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.183922 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.184063 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260811 10:07:32.194036 29396 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:32.194808 29396 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:32.199115 29396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7f72d21fa194e518e43149647b62182: Generated new TSK 0
I20260811 10:07:32.199342 29396 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:32.234216 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37467
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:32.326751 29411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:32.326969 29411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:32.326993 29411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:32.328102 29411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:32.328140 29411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:32.328166 29411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:32.329638 29411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:32.329834 29411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:32.330072 29411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:32.335570 29411 server_base.cc:1034] running on GCE node
W20260811 10:07:32.335470 29419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.335471 29417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.335469 29416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:32.336010 29411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:32.336293 29411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:32.337455 29411 hybrid_clock.cc:648] HybridClock initialized: now 1786442852337440 us; error 28 us; skew 500 ppm
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.340448 29411 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:32.340963 29411 webserver.cc:460] Webserver started at http://127.22.177.1:39875/ using document root <none> and password file <none>
I20260811 10:07:32.341214 29411 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:32.341295 29411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:32.341449 29411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:32.342590 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "eaae83a5a8c646ae928455b75c8d81c9"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "2c82fc3114293ec56c5e047065981fab"
server_key_iv: "9cb7ac210f7577de9514e25d6b5ff7cd"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.342955 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "eaae83a5a8c646ae928455b75c8d81c9"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "2c82fc3114293ec56c5e047065981fab"
server_key_iv: "9cb7ac210f7577de9514e25d6b5ff7cd"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.344439 29411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:32.345393 29426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.345674 29411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:32.345794 29411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "eaae83a5a8c646ae928455b75c8d81c9"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "2c82fc3114293ec56c5e047065981fab"
server_key_iv: "9cb7ac210f7577de9514e25d6b5ff7cd"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.345892 29411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:32.365006 29411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:32.367625 29411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:32.367825 29411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:32.368125 29411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:32.368507 29411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:32.368577 29411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.368630 29411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:32.368682 29411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.375820 29411 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:38747
I20260811 10:07:32.375891 29549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:38747 every 8 connection(s)
I20260811 10:07:32.376251 29411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:32.380870 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29411
I20260811 10:07:32.381012 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:32.381222 23236 external_mini_cluster.cc:1428] Setting key 06a8d61b3e0314ef46742e5a4fb23581
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:32.387696 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.377048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:59157 (local address 127.22.177.62:37467)
0811 10:07:32.377121 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.377123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.377345 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:07:32.378512 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.378513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.378524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.378583 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.379343 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.379910 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.380963 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.381293 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.381377 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.384190 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.384191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.384191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.384198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.385742 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.386313 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.386315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.386315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.386376 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.386733 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.386735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.386735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.387037 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.387100 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.387518 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.387617 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 10:07:32.387974 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.377018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37467 (local address 127.22.177.1:59157)
0811 10:07:32.377204 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:32.377248 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:32.378409 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:32.378700 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:32.378703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:32.378986 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:32.379178 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:32.379231 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.379998 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.379999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:32.380865 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:32.380868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.381420 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.381422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:32.381636 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.382216 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:32.382225 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:32.384065 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:32.385854 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:32.385862 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:32.385865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:32.386210 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:32.386460 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:32.386461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:32.386461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:32.386631 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:32.387121 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:32.387129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:32.387330 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 10:07:32.387463 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 10:07:32.387665 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260811 10:07:32.388550 29550 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37467
I20260811 10:07:32.388641 29550 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:32.388841 29550 heartbeater.cc:511] Master 127.22.177.62:37467 requested a full tablet report, sending...
I20260811 10:07:32.389396 29342 ts_manager.cc:194] Registered new tserver with Master: eaae83a5a8c646ae928455b75c8d81c9 (127.22.177.1:38747)
I20260811 10:07:32.390084 29342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:59157
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:32.413075 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37467
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:32.500379 29555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:32.500662 29555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:32.500712 29555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:32.501924 29555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:32.502005 29555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:32.502051 29555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:32.503705 29555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:32.504019 29555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:32.504299 29555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:32.510838 29561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.510818 29562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.510824 29564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:32.511497 29555 server_base.cc:1034] running on GCE node
I20260811 10:07:32.511716 29555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:32.511970 29555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:32.513142 29555 hybrid_clock.cc:648] HybridClock initialized: now 1786442852513129 us; error 30 us; skew 500 ppm
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.516031 29555 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:32.516538 29555 webserver.cc:460] Webserver started at http://127.22.177.2:45791/ using document root <none> and password file <none>
I20260811 10:07:32.516775 29555 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:32.516847 29555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:32.517007 29555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:32.518218 29555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "6871c5aa354e486ca7f8c5b236da53fb"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "8fcacf6ddebb724d823370bd95696c56"
server_key_iv: "1c2dd86f4d8c082817c0c7c03f61ab3c"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.518585 29555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "6871c5aa354e486ca7f8c5b236da53fb"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "8fcacf6ddebb724d823370bd95696c56"
server_key_iv: "1c2dd86f4d8c082817c0c7c03f61ab3c"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.520045 29555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:07:32.520972 29572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.521240 29555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:32.521366 29555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "6871c5aa354e486ca7f8c5b236da53fb"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "8fcacf6ddebb724d823370bd95696c56"
server_key_iv: "1c2dd86f4d8c082817c0c7c03f61ab3c"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.521471 29555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:32.550772 29555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:32.553370 29555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:32.553548 29555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:32.553850 29555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:32.554186 29555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:32.554252 29555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.554296 29555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:32.554327 29555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.560451 29555 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:41249
I20260811 10:07:32.560534 29693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:41249 every 8 connection(s)
I20260811 10:07:32.560886 29555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:32.569963 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29555
I20260811 10:07:32.570106 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:32.570361 23236 external_mini_cluster.cc:1428] Setting key a5e0e547f4915867a8195a97bf43467c
I20260811 10:07:32.572046 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.561766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:60939 (local address 127.22.177.62:37467)
0811 10:07:32.561843 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.561846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.562139 (+ 293us) server_negotiation.cc:407] Connection header received
0811 10:07:32.563138 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.563139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.563151 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.563285 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.563927 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.564489 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.565636 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.565848 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.565914 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.568421 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.568423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.568423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.568431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.570077 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.570774 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.570776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.570776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.570854 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.571344 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.571345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.571346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.571483 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.571542 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.571897 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.571980 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:07:32.572382 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.561749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37467 (local address 127.22.177.2:60939)
0811 10:07:32.561983 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:32.562024 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:32.563023 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:32.563351 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:32.563354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:32.563629 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:32.563833 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:32.563837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.564597 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.564598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:32.565478 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:32.565481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.565958 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.565959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:32.566084 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.566689 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:32.566699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:32.568315 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:32.570286 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:32.570295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:32.570298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:32.570630 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:32.570974 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:32.570975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:32.570975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:32.571168 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:32.571559 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:32.571567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:32.571746 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 10:07:32.571868 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 10:07:32.572066 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
I20260811 10:07:32.573154 29694 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37467
I20260811 10:07:32.573280 29694 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:32.573446 29694 heartbeater.cc:511] Master 127.22.177.62:37467 requested a full tablet report, sending...
I20260811 10:07:32.573902 29342 ts_manager.cc:194] Registered new tserver with Master: 6871c5aa354e486ca7f8c5b236da53fb (127.22.177.2:41249)
I20260811 10:07:32.574434 29342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:60939
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:32.602552 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37467
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:32.690774 29699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:32.690985 29699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:32.691010 29699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:32.692164 29699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:32.692209 29699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:32.692235 29699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:32.693840 29699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:32.694088 29699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:32.694326 29699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:32.699982 29707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.700089 29704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:32.699982 29705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:32.700466 29699 server_base.cc:1034] running on GCE node
I20260811 10:07:32.700724 29699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:32.701049 29699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:32.702319 29699 hybrid_clock.cc:648] HybridClock initialized: now 1786442852702293 us; error 46 us; skew 500 ppm
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.704962 29699 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:32.705546 29699 webserver.cc:460] Webserver started at http://127.22.177.3:41063/ using document root <none> and password file <none>
I20260811 10:07:32.705832 29699 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:32.705909 29699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:32.706068 29699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:32.707175 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "500bdb5eab0c4933bd46e1cc4554be74"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "fb90c61def43a222497492ec1adebaa8"
server_key_iv: "21431a0544089f741de79f98c2838d31"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.707566 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "500bdb5eab0c4933bd46e1cc4554be74"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "fb90c61def43a222497492ec1adebaa8"
server_key_iv: "21431a0544089f741de79f98c2838d31"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.709139 29699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:32.710111 29716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.710387 29699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:32.710511 29699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "500bdb5eab0c4933bd46e1cc4554be74"
format_stamp: "Formatted at 2026-08-11 10:07:32 on dist-test-slave-g350"
server_key: "fb90c61def43a222497492ec1adebaa8"
server_key_iv: "21431a0544089f741de79f98c2838d31"
server_key_version: "encryptionkey@0"
I20260811 10:07:32.710603 29699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:32.741484 29699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:32.743856 29699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:32.744072 29699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:32.744421 29699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:32.744781 29699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:32.744840 29699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.744886 29699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:32.744911 29699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:32.751577 29699 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:34149
I20260811 10:07:32.751621 29841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:34149 every 8 connection(s)
I20260811 10:07:32.751965 29699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:32.759294 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29699
I20260811 10:07:32.759402 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:32.759605 23236 external_mini_cluster.cc:1428] Setting key d1baec37c5698808635eb8c630f49082
I20260811 10:07:32.762714 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.752846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52267 (local address 127.22.177.62:37467)
0811 10:07:32.752912 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.752914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.753105 (+ 191us) server_negotiation.cc:407] Connection header received
0811 10:07:32.754182 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.754183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.754193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.754259 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.754903 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.755450 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.756512 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.756730 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.756799 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.759405 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.759407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.759407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.759417 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.761157 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.761661 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.761662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.761663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.761754 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.762072 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.762073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.762073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.762193 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.762260 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.762571 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.762645 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:07:32.762881 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.752804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37467 (local address 127.22.177.3:52267)
0811 10:07:32.752974 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:32.753016 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:32.754082 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:32.754313 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:32.754316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:32.754569 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:32.754779 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:32.754784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.755571 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.755572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:32.756415 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:32.756418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.756846 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.756847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:32.756957 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.757523 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:32.757532 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:32.759265 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:32.761275 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:32.761282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:32.761284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:32.761540 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:32.761850 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:32.761851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:32.761851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:32.761986 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:32.762287 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:32.762293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:32.762431 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 10:07:32.762516 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:07:32.762663 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260811 10:07:32.763463 29842 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37467
I20260811 10:07:32.763553 29842 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:32.763705 29842 heartbeater.cc:511] Master 127.22.177.62:37467 requested a full tablet report, sending...
I20260811 10:07:32.764120 29342 ts_manager.cc:194] Registered new tserver with Master: 500bdb5eab0c4933bd46e1cc4554be74 (127.22.177.3:34149)
I20260811 10:07:32.764506 29342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:52267
I20260811 10:07:32.772225 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/test-admin.keytab.
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.791126 23236 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 10:07:32.793150 29849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:32.792148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35178 (local address 127.22.177.1:38747)
0811 10:07:32.792422 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.792426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.792436 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:32.792463 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.792464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.792479 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.792579 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.793047 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35178: BlockingRecv error: recv got EOF from 127.0.0.1:35178 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":137,"threads_started":1}
W20260811 10:07:32.793284 23236 init.cc:244] Error while opening credential cache '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc)
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.794445 23236 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.795531 23236 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:32.804477 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.796410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.22.177.1:38747)
0811 10:07:32.796468 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.796470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.796477 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:07:32.796498 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.796499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.796508 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.796577 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.797015 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.797868 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.798632 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.798966 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.799051 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.801353 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.801362 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.801363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.801370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.803149 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.803607 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.803611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.803611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.803673 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.803887 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.803888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.803888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.804038 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.804198 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.804265 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.804352 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 10:07:32.807979 29849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:32.807078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35186 (local address 127.22.177.1:38747)
0811 10:07:32.807129 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.807131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.807140 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:32.807188 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.807189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.807199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.807275 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.807920 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35186: BlockingRecv error: recv got EOF from 127.0.0.1:35186 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 10:07:32.808133 23236 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc' of type FILE
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.809389 23236 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.810575 23236 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:32.818238 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.811162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35198 (local address 127.22.177.1:38747)
0811 10:07:32.811224 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.811225 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.811253 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:07:32.811319 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.811320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.811361 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.811431 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.811793 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.812330 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.813030 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.813308 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.813372 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.815641 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.815642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.815642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.815649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.817063 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.817399 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.817400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.817401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.817460 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.817800 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.817801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.817801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.817914 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.818004 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.818101 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.818170 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 10:07:32.819800 29849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:32.819364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.22.177.1:38747)
0811 10:07:32.819400 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.819402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.819434 (+ 32us) server_negotiation.cc:407] Connection header received
0811 10:07:32.819481 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.819481 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.819490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.819537 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.819745 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35206: BlockingRecv error: recv got EOF from 127.0.0.1:35206 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260811 10:07:32.820098 23236 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786442825609099-23236-0/krb5kdc/krb5cc' of type FILE
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.821373 23236 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:32.822383 23236 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:07:32.830835 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:32.823042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.22.177.1:38747)
0811 10:07:32.823105 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:32.823107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:32.823122 (+ 15us) server_negotiation.cc:407] Connection header received
0811 10:07:32.823140 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:32.823141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:32.823149 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:32.823216 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:32.823693 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.824174 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.824966 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:32.825289 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:32.825361 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:32.827859 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:32.827860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:32.827860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:32.827868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:32.829392 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.829990 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.829991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.829991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.830060 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:32.830325 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:32.830325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:32.830326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:32.830466 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:32.830570 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:32.830687 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 10:07:32.830762 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:07:32.831702 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29411
I20260811 10:07:32.843042 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29555
I20260811 10:07:32.849954 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29699
I20260811 10:07:32.858947 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29307
2026-08-11T10:07:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2109 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29862](info): set up 2 sockets
Aug 11 10:07:32 dist-test-slave-g350 krb5kdc[29862](info): commencing operation
krb5kdc: starting...
W20260811 10:07:34.045889 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 10:07:34.243359 23236 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.197s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 10:07:34.391911 23236 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.148s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 10:07:34.591239 23236 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.199s user 0.002s sys 0.000s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/test-user.keytab.
W20260811 10:07:34.860777 23236 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.269s user 0.002s sys 0.000s
Aug 11 10:07:34 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab.
Entry for principal oryx/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab.
I20260811 10:07:34.913883 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37093
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab
--principal=oryx/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:35.000618 29876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:35.000840 29876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:35.000864 29876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:35.002028 29876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:35.002068 29876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:35.002082 29876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:35.002095 29876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:35.002107 29876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:35.003636 29876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab
--principal=oryx/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:35.003840 29876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:35.004014 29876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:35.009492 29885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.009492 29882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.009758 29883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.009747 29876 server_base.cc:1034] running on GCE node
I20260811 10:07:35.010144 29876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:35.010485 29876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:35.011660 29876 hybrid_clock.cc:648] HybridClock initialized: now 1786442855011627 us; error 57 us; skew 500 ppm
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:35.017891 29876 init.cc:377] Logged in from keytab as oryx/127.22.177.62@KRBTEST.COM (short username oryx)
I20260811 10:07:35.018431 29876 webserver.cc:460] Webserver started at http://127.22.177.62:38759/ using document root <none> and password file <none>
I20260811 10:07:35.018676 29876 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.018724 29876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.018909 29876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:35.020180 29876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "0e167745826951d831a08bda8691a529"
server_key_iv: "858fe26148f41044bfc0b9d5c605dd46"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.020567 29876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "0e167745826951d831a08bda8691a529"
server_key_iv: "858fe26148f41044bfc0b9d5c605dd46"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.022084 29876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:35.022946 29893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.023326 29876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:35.023456 29876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "0e167745826951d831a08bda8691a529"
server_key_iv: "858fe26148f41044bfc0b9d5c605dd46"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.023545 29876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.038796 29876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:35.040910 29876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:35.041093 29876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:35.045524 29876 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37093
I20260811 10:07:35.045563 29952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37093 every 8 connection(s)
I20260811 10:07:35.045995 29876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:35.046991 29955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:35.049324 29955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Bootstrap starting.
I20260811 10:07:35.049973 29955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:35.050004 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29876
I20260811 10:07:35.050299 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:35.050287 29955 log.cc:824] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:35.050503 23236 external_mini_cluster.cc:1428] Setting key 243c5d6fa8437bf21b8aa1f0acbb8f03
I20260811 10:07:35.051160 29955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: No bootstrap required, opened a new log
I20260811 10:07:35.053351 29955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } }
I20260811 10:07:35.053511 29955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:35.053560 29955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5a9fbb091449f0bf7ff897bd7d6c8a, State: Initialized, Role: FOLLOWER
I20260811 10:07:35.053712 29955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } }
I20260811 10:07:35.053814 29955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:35.053838 29955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:35.053903 29955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:35.054451 29955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } }
I20260811 10:07:35.054538 29955 leader_election.cc:302] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f5a9fbb091449f0bf7ff897bd7d6c8a; no voters:
I20260811 10:07:35.054747 29955 leader_election.cc:288] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:35.054795 29960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:35.054967 29960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 LEADER]: Becoming Leader. State: Replica: 3f5a9fbb091449f0bf7ff897bd7d6c8a, State: Running, Role: LEADER
I20260811 10:07:35.055012 29955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:35.055099 29960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } }
I20260811 10:07:35.055439 29960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f5a9fbb091449f0bf7ff897bd7d6c8a. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } }
I20260811 10:07:35.055680 29960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:35.055608 29961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } }
I20260811 10:07:35.055982 29961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:35.056161 29969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:35.057091 29969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:35.058529 29969 catalog_manager.cc:1347] Generated new cluster ID: 1dc09f59a80048908d229b2985f3ca62
I20260811 10:07:35.058581 29969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:35.062017 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.051064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.22.177.62:37093)
0811 10:07:35.051288 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.051298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.051329 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:07:35.051724 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.051730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.051774 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.051918 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.052542 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.053422 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.054327 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.054947 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.055423 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.058158 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.058162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.058162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.058174 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.060332 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.060737 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.060741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.060741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.060854 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.061063 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.061064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.061064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.061379 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.061543 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.061650 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.061792 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260811 10:07:35.084113 29969 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:35.084905 29969 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:35.089942 29969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Generated new TSK 0
I20260811 10:07:35.090157 29969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab.
Entry for principal oryx/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab.
I20260811 10:07:35.125676 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37093
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:35.212436 29981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:35.212697 29981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:35.212738 29981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:35.213939 29981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:35.214016 29981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:35.214064 29981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:35.215585 29981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:35.215879 29981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:35.216131 29981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:35.221664 29989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.221688 29986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.221760 29981 server_base.cc:1034] running on GCE node
W20260811 10:07:35.221642 29987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.222206 29981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:35.222530 29981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:35.223747 29981 hybrid_clock.cc:648] HybridClock initialized: now 1786442855223729 us; error 47 us; skew 500 ppm
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:35.226799 29981 init.cc:377] Logged in from keytab as oryx/127.22.177.1@KRBTEST.COM (short username oryx)
I20260811 10:07:35.227330 29981 webserver.cc:460] Webserver started at http://127.22.177.1:36185/ using document root <none> and password file <none>
I20260811 10:07:35.227628 29981 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.227700 29981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.227874 29981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:35.228978 29981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "ae22e677e6f9c5704c4eedd52aa3b69e"
server_key_iv: "ff3eaf8d7b45baf6cb8b80177065746e"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.229359 29981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "ae22e677e6f9c5704c4eedd52aa3b69e"
server_key_iv: "ff3eaf8d7b45baf6cb8b80177065746e"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.230876 29981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:35.231701 29996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.231887 29981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:35.231998 29981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "ae22e677e6f9c5704c4eedd52aa3b69e"
server_key_iv: "ff3eaf8d7b45baf6cb8b80177065746e"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.232090 29981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.246809 29981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:35.249418 29981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:35.249680 29981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:35.249982 29981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:35.250356 29981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:35.250419 29981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.250465 29981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:35.250514 29981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.257666 29981 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:40717
I20260811 10:07:35.257689 30119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:40717 every 8 connection(s)
I20260811 10:07:35.258071 29981 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:35.262162 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 29981
I20260811 10:07:35.262266 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:35.262501 23236 external_mini_cluster.cc:1428] Setting key 8408cc5dccd3ef5a6664c7ff00899cb4
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:35.269177 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.259084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:42011 (local address 127.22.177.62:37093)
0811 10:07:35.259136 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.259138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.259339 (+ 201us) server_negotiation.cc:407] Connection header received
0811 10:07:35.259834 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.259836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.259847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.259921 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.260971 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.261526 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.262731 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.262991 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.263077 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.265980 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.265982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.265982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.265989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.267539 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.268039 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.268040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.268041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.268103 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.268371 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.268372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.268373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.268536 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.268612 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.268930 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.269037 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:07:35.269200 30122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.258998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.1:42011)
0811 10:07:35.259170 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.259226 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.259720 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.259973 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.259976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.260667 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.260871 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.260876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.261677 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.261677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.262660 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.262663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.263092 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.263094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.263227 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.263787 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.263798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.265875 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.267651 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.267660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.267663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.267944 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.268181 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.268182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.268182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.268299 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.268621 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.268628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.268774 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.268877 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.269013 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260811 10:07:35.269891 30120 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37093
I20260811 10:07:35.270009 30120 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:35.270229 30120 heartbeater.cc:511] Master 127.22.177.62:37093 requested a full tablet report, sending...
I20260811 10:07:35.270836 29912 ts_manager.cc:194] Registered new tserver with Master: ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717)
I20260811 10:07:35.271538 29912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:42011
WARNING: no policy specified for oryx/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab.
Entry for principal oryx/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab.
I20260811 10:07:35.294241 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37093
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:35.380712 30125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:35.380959 30125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:35.380999 30125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:35.382190 30125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:35.382274 30125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:35.382323 30125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:35.383841 30125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:35.384112 30125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:35.384358 30125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:35.390177 30134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.390177 30132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.390336 30125 server_base.cc:1034] running on GCE node
W20260811 10:07:35.390475 30131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.390715 30125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:35.391050 30125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:35.392288 30125 hybrid_clock.cc:648] HybridClock initialized: now 1786442855392235 us; error 65 us; skew 500 ppm
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:35.395087 30125 init.cc:377] Logged in from keytab as oryx/127.22.177.2@KRBTEST.COM (short username oryx)
I20260811 10:07:35.395648 30125 webserver.cc:460] Webserver started at http://127.22.177.2:36449/ using document root <none> and password file <none>
I20260811 10:07:35.395939 30125 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.396018 30125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.396181 30125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:35.397373 30125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "8b1262c1389b4425a6a42cb6df58fae6"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "f343667d7cfb79cfa91c1ccb923e5505"
server_key_iv: "bc7ace6e7717d644b6c2621679ffd65e"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.397804 30125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "8b1262c1389b4425a6a42cb6df58fae6"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "f343667d7cfb79cfa91c1ccb923e5505"
server_key_iv: "bc7ace6e7717d644b6c2621679ffd65e"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.399394 30125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:35.400321 30142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.400516 30125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:35.400619 30125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "8b1262c1389b4425a6a42cb6df58fae6"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "f343667d7cfb79cfa91c1ccb923e5505"
server_key_iv: "bc7ace6e7717d644b6c2621679ffd65e"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.400710 30125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.415525 30125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:35.418206 30125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:35.418396 30125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:35.418681 30125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:35.419050 30125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:35.419114 30125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.419162 30125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:35.419190 30125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.425935 30125 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:41947
I20260811 10:07:35.426025 30263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:41947 every 8 connection(s)
I20260811 10:07:35.426440 30125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:35.430778 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 30125
I20260811 10:07:35.430920 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:35.431108 23236 external_mini_cluster.cc:1428] Setting key d9694c5756d153e5833636e1b8147f2f
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:35.438107 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.427316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:56461 (local address 127.22.177.62:37093)
0811 10:07:35.427378 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.427380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.427614 (+ 234us) server_negotiation.cc:407] Connection header received
0811 10:07:35.428180 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.428181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.428195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.428264 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.429256 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.429893 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.431204 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.431523 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.431603 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.434330 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.434332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.434332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.434343 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.436285 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.436783 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.436785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.436786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.436870 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.437176 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.437178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.437178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.437388 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.437482 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.437884 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.437995 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:07:35.438342 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.427322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:56461)
0811 10:07:35.427483 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.427519 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.428018 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.428336 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.428339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.428942 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.429143 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.429148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.430017 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.430018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.431097 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.431100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.431628 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.431629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.431749 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.432371 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.432382 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.434204 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.436400 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.436410 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.436413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.436690 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.436959 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.436960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.436961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.437094 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.437469 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.437478 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.437707 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.437857 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.438070 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260811 10:07:35.438880 30264 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37093
I20260811 10:07:35.438982 30264 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:35.439205 30264 heartbeater.cc:511] Master 127.22.177.62:37093 requested a full tablet report, sending...
I20260811 10:07:35.439687 29912 ts_manager.cc:194] Registered new tserver with Master: 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947)
I20260811 10:07:35.440258 29912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:56461
WARNING: no policy specified for oryx/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab.
Entry for principal oryx/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab.
I20260811 10:07:35.465695 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37093
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:35.552016 30269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:35.552239 30269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:35.552266 30269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:35.553475 30269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:35.553521 30269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:35.553550 30269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:35.555222 30269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:35.555518 30269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:35.555825 30269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:35.561865 30269 server_base.cc:1034] running on GCE node
W20260811 10:07:35.561776 30275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.561738 30277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.561738 30274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.562626 30269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:35.562981 30269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:35.564158 30269 hybrid_clock.cc:648] HybridClock initialized: now 1786442855564147 us; error 30 us; skew 500 ppm
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:35.566901 30269 init.cc:377] Logged in from keytab as oryx/127.22.177.3@KRBTEST.COM (short username oryx)
I20260811 10:07:35.567340 30269 webserver.cc:460] Webserver started at http://127.22.177.3:45105/ using document root <none> and password file <none>
I20260811 10:07:35.567600 30269 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.567672 30269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.567833 30269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:35.568913 30269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "de5bedd438b448b9992a005cdb498bb9"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "dd55db31987e86a760d79c0960bfc440"
server_key_iv: "54013d0fd64ef4d689b9d9cfaf74934c"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.569290 30269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "de5bedd438b448b9992a005cdb498bb9"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "dd55db31987e86a760d79c0960bfc440"
server_key_iv: "54013d0fd64ef4d689b9d9cfaf74934c"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.570757 30269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:35.571609 30284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.571818 30269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:35.571930 30269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "de5bedd438b448b9992a005cdb498bb9"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "dd55db31987e86a760d79c0960bfc440"
server_key_iv: "54013d0fd64ef4d689b9d9cfaf74934c"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.572006 30269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.591969 30269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:35.594568 30269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:35.594726 30269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:35.595050 30269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:35.595413 30269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:35.595471 30269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.595513 30269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:35.595546 30269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.601778 30269 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:41367
I20260811 10:07:35.601895 30405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:41367 every 8 connection(s)
I20260811 10:07:35.602200 30269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:35.611840 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.602974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52379 (local address 127.22.177.62:37093)
0811 10:07:35.603042 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.603044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.603224 (+ 180us) server_negotiation.cc:407] Connection header received
0811 10:07:35.603770 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.603771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.603786 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.603836 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.604892 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.605478 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.606532 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.606732 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.606812 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.609003 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.609004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.609004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.609012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.610502 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.610908 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.610908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.610909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.610967 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.611190 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.611191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.611191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.611347 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.611403 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.611690 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.611772 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:07:35.611967 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.602933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:52379)
0811 10:07:35.603096 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.603139 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.603647 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.603901 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.603906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.604582 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.604776 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.604781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.605579 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.605580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.606432 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.606435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.606822 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.606823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.606961 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.607490 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.607499 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.608908 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.610588 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.610595 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.610596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.610831 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.611032 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.611033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.611033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.611132 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.611411 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.611417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.611535 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.611622 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.611745 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260811 10:07:35.612207 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 30269
I20260811 10:07:35.612304 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:35.612521 23236 external_mini_cluster.cc:1428] Setting key f77ff11bb254ac8d4afdb6234a95ee6a
I20260811 10:07:35.612581 30406 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37093
I20260811 10:07:35.612687 30406 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:35.612923 30406 heartbeater.cc:511] Master 127.22.177.62:37093 requested a full tablet report, sending...
I20260811 10:07:35.613498 29912 ts_manager.cc:194] Registered new tserver with Master: de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367)
I20260811 10:07:35.613927 29912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:52379
I20260811 10:07:35.624749 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.177.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.61.keytab.
Entry for principal oryx/127.22.177.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.61.keytab.
I20260811 10:07:35.657107 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:44937
--webserver_interface=127.22.177.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.61.keytab
--principal=oryx/127.22.177.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37093,127.22.177.61:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:35.743040 30411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:35.743294 30411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:35.743335 30411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:35.744604 30411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:35.744701 30411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:35.744731 30411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:35.744762 30411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:35.744786 30411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:35.746361 30411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37093,127.22.177.61:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.61.keytab
--principal=oryx/127.22.177.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.177.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
W20260811 10:07:35.746510 30411 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.177.62:37093,127.22.177.61:44937'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 10:07:35.746678 30411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:35.746914 30411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:35.752179 30416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.752414 30411 server_base.cc:1034] running on GCE node
W20260811 10:07:35.752550 30417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.752614 30419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.752918 30411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:35.753232 30411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:35.754427 30411 hybrid_clock.cc:648] HybridClock initialized: now 1786442855754409 us; error 44 us; skew 500 ppm
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:35.757347 30411 init.cc:377] Logged in from keytab as oryx/127.22.177.61@KRBTEST.COM (short username oryx)
I20260811 10:07:35.757907 30411 webserver.cc:460] Webserver started at http://127.22.177.61:43599/ using document root <none> and password file <none>
I20260811 10:07:35.758154 30411 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.758225 30411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.758419 30411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:35.759639 30411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/instance:
uuid: "369c82f45c084c0ba7e646b970b38f1e"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "09723070c9ac66707e2c41a2a066c8ef"
server_key_iv: "4e0eebdd0c1e1142c24b09adb37018ae"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.760026 30411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal/instance:
uuid: "369c82f45c084c0ba7e646b970b38f1e"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "09723070c9ac66707e2c41a2a066c8ef"
server_key_iv: "4e0eebdd0c1e1142c24b09adb37018ae"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.761554 30411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:35.762435 30426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.762642 30411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:35.762753 30411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
uuid: "369c82f45c084c0ba7e646b970b38f1e"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "09723070c9ac66707e2c41a2a066c8ef"
server_key_iv: "4e0eebdd0c1e1142c24b09adb37018ae"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.762838 30411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.775635 30411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:35.778280 30411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:35.778465 30411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:35.782677 30411 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.61:44937
I20260811 10:07:35.782723 30484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.61:44937 every 8 connection(s)
I20260811 10:07:35.783114 30411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
I20260811 10:07:35.783397 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 30411
I20260811 10:07:35.784605 30485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:35.786892 30485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.61@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:35.795778 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.784151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.22.177.61:44937)
0811 10:07:35.784411 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.784421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.784458 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:07:35.785142 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.785152 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.785222 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.785395 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.786178 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.787645 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.788640 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.789116 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.789357 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.791704 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.791712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.791712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.791720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.793784 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.794287 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.794291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.794291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.794387 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.794616 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.794616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.794617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.794871 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.794991 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.795229 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.795511 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":119,"threads_started":1}
I20260811 10:07:35.796231 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.787689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57188 (local address 127.22.177.62:37093)
0811 10:07:35.787763 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.787766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.787927 (+ 161us) server_negotiation.cc:407] Connection header received
0811 10:07:35.788003 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.788005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.788017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.788106 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.788619 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.789162 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.790162 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.790383 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.790469 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.792961 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.792963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.792963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.792971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.794691 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.795183 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.795185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.795185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.795267 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.795494 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.795496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.795496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.795709 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.795796 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.796011 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.796109 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:07:35.796250 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.787634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:57188)
0811 10:07:35.787823 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.787870 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.787940 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.788126 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.788130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.788436 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.788512 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.788513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.789275 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.789276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.790062 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.790064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.790486 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.790487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.790570 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.791102 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.791111 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.792885 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.794767 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.794774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.794776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.795075 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.795361 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.795362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.795362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.795417 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.795788 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.795793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.795856 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.795975 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.796084 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260811 10:07:35.796823 30485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.61@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
I20260811 10:07:35.803622 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.797096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:53750)
0811 10:07:35.797170 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.797182 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.797245 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.797484 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.797485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.797681 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.797721 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.797723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.798374 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.798375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.798880 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.798881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.799265 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.799266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.799320 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.799707 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.799711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.800966 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.802567 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.802568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.802569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.802740 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.802953 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.802953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.802954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.803008 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.803255 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.803256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.803295 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.803363 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.803486 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 10:07:35.803622 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.797340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.22.177.61:44937)
0811 10:07:35.797383 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.797385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.797390 (+ 5us) server_negotiation.cc:407] Connection header received
0811 10:07:35.797403 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.797403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.797410 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.797466 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.797798 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.798289 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.798968 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.799178 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.799264 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.801081 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.801082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.801082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.801089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.802464 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.802816 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.802817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.802817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.802875 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.803074 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.803075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.803076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.803181 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.803263 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.803405 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.803520 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":24}
I20260811 10:07:35.805825 30485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Bootstrap starting.
I20260811 10:07:35.806586 30485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:35.806842 30485 log.cc:824] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:35.807770 30485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: No bootstrap required, opened a new log
I20260811 10:07:35.809299 30485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } }
I20260811 10:07:35.809435 30485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:35.809463 30485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369c82f45c084c0ba7e646b970b38f1e, State: Initialized, Role: FOLLOWER
I20260811 10:07:35.809563 30485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } }
I20260811 10:07:35.809937 30492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } } }
I20260811 10:07:35.810000 30485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:35.810047 30492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:07:35.811923 30504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:35.812023 30504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:35.814244 30411 master_runner.cc:428] Detected that this master 369c82f45c084c0ba7e646b970b38f1e is joining an existing cluster
I20260811 10:07:35.814347 30411 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.177.61:44937
I20260811 10:07:35.823665 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.815727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.22.177.62:37093)
0811 10:07:35.815807 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.815810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.815820 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:35.815846 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.815847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.815858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.815940 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.816363 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.817201 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.818385 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.818687 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.818734 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.820111 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.820113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.820113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.820121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.822239 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.822734 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.822735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.822735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.822821 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.823029 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.823030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.823030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.823194 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.823265 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.823464 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.823551 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 10:07:35.823757 30511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.815513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:53754)
0811 10:07:35.816041 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.816051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.816112 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.816270 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.816271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.816481 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.816542 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.816544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.817467 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.817468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.818258 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.818261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.818685 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.818686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.818756 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.819286 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.819291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.819774 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.822484 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.822485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.822486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.822744 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.822944 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.822945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.822945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.823006 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.823267 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.823268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.823332 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.823425 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.823591 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260811 10:07:35.824126 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.815512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:57194)
0811 10:07:35.815700 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.815711 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.815764 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.815961 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.815962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.816149 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.816262 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.816264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.817297 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.817298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.818274 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.818277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.818816 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.818817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.818882 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.819507 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.819512 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.820009 (+ 497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.822339 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.822340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.822341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.822642 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.822882 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.822883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.822883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.822951 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.823295 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.823296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.823349 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.823815 (+ 466us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.824015 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":129,"threads_started":1}
I20260811 10:07:35.824241 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.815763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.22.177.61:44937)
0811 10:07:35.815994 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.815996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.816092 (+ 96us) server_negotiation.cc:407] Connection header received
0811 10:07:35.816182 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.816182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.816190 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.816216 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.816633 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.817372 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.818376 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.818575 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.818667 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.819886 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.819887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.819888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.819895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.822390 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.822824 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.822825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.822825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.822886 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.823063 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.823064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.823064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.823185 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.823276 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.823467 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.823653 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":211}
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.62@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
I20260811 10:07:35.835335 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.826895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.22.177.61:44937)
0811 10:07:35.826934 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.826936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.827112 (+ 176us) server_negotiation.cc:407] Connection header received
0811 10:07:35.827267 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.827269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.827280 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.827343 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.827807 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.828331 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.829188 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.829416 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.829500 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.831848 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.831849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.831850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.831856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.833730 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.834218 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.834219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.834220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.834345 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.834634 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.834635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.834635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.834765 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.834851 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.835082 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.835218 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:07:35.835353 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.826826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:53762)
0811 10:07:35.827005 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.827042 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.827140 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.827350 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.827354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.827660 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.827713 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.827715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.828431 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.828432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.829090 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.829092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.829512 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.829513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.829649 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.830184 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.830195 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.831750 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.833813 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.833822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.833824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.834101 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.834461 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.834462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.834463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.834546 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.834857 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.834866 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.834931 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.835051 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.835222 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260811 10:07:35.836408 29912 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.177.61:44937: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e"
member_type: NON_VOTER
last_known_addr {
host: "127.22.177.61"
port: 44937
}
attrs {
promote: true
}
}
dest_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a"
cas_config_opid_index: -1
I20260811 10:07:35.836789 29912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } }
I20260811 10:07:35.837347 30514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 369c82f45c084c0ba7e646b970b38f1e (127.22.177.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } }
I20260811 10:07:35.837846 30455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:35.838483 30514 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.177.61:44937
I20260811 10:07:35.838572 30514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } }
I20260811 10:07:35.838639 30514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:35.838768 30455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Refusing update from remote peer 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 10:07:35.839174 30513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:35.839342 30411 master.cc:537] Master@127.22.177.61:44937 shutting down...
W20260811 10:07:35.839953 29894 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 10:07:35.840474 29894 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a -> Peer 369c82f45c084c0ba7e646b970b38f1e (127.22.177.61:44937): Couldn't send request to peer 369c82f45c084c0ba7e646b970b38f1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 10:07:35.842360 30411 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:35.842470 30411 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:35.842528 30411 tablet_replica.cc:330] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: stopping tablet replica
I20260811 10:07:35.855829 30411 master.cc:552] Master@127.22.177.61:44937 shutdown complete.
I20260811 10:07:35.855903 30411 master_runner.cc:305] Clearing existing system tablet
I20260811 10:07:35.856057 30411 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.856110 30411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.856958 30411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:35.857648 30519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.857802 30411 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.857869 30411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:35.857908 30411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
uuid: "369c82f45c084c0ba7e646b970b38f1e"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "09723070c9ac66707e2c41a2a066c8ef"
server_key_iv: "4e0eebdd0c1e1142c24b09adb37018ae"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.858613 30411 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:35.860301 30411 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 10:07:35.860390 30411 log.cc:1196] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 10:07:35.860592 30411 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Deleting consensus metadata
I20260811 10:07:35.860720 30411 master_runner.cc:315] Copying system tablet from 127.22.177.62:37093
I20260811 10:07:35.861424 30411 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: tablet copy: Beginning tablet copy session from remote peer at address 127.22.177.62:37093
I20260811 10:07:35.868503 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.861856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:57206)
0811 10:07:35.862026 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:35.862036 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:35.862114 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:35.862262 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:35.862262 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:35.862442 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:35.862502 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.862504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.863479 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.863480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.864046 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:35.864048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.864540 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.864541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:35.864595 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.865174 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:35.865179 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:35.865522 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:35.867474 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.867475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.867476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.867654 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.867870 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:35.867871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:35.867871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:35.867927 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:35.868205 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:35.868206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:35.868255 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:07:35.868332 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:07:35.868423 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260811 10:07:35.868563 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.861949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.22.177.62:37093)
0811 10:07:35.862006 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.862009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.862114 (+ 105us) server_negotiation.cc:407] Connection header received
0811 10:07:35.862167 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.862169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.862181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.862252 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.862584 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.863380 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.864143 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.864454 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.864530 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.865642 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.865643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.865644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.865652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.867381 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.867732 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.867733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.867734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.867800 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.867991 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.867992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.867992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.868138 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.868212 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.868383 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.868469 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:07:35.869009 29936 tablet_copy_service.cc:140] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 369c82f45c084c0ba7e646b970b38f1e ({username='oryx', principal='oryx/127.22.177.61@KRBTEST.COM'} at 127.0.0.1:57206)
I20260811 10:07:35.869119 29936 tablet_copy_service.cc:161] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 369c82f45c084c0ba7e646b970b38f1e at {username='oryx', principal='oryx/127.22.177.61@KRBTEST.COM'} at 127.0.0.1:57206: session id = 369c82f45c084c0ba7e646b970b38f1e-00000000000000000000000000000000
I20260811 10:07:35.870033 29936 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 10:07:35.870702 30411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 10:07:35.871724 30411 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: tablet copy: Starting download of 0 data blocks...
I20260811 10:07:35.871905 30411 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: tablet copy: Starting download of 1 WAL segments...
I20260811 10:07:35.873163 30411 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 10:07:35.874217 29936 tablet_copy_service.cc:342] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Request end of tablet copy session 369c82f45c084c0ba7e646b970b38f1e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.177.61@KRBTEST.COM'} at 127.0.0.1:57206
I20260811 10:07:35.874274 29936 tablet_copy_service.cc:433] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: ending tablet copy session 369c82f45c084c0ba7e646b970b38f1e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 369c82f45c084c0ba7e646b970b38f1e
I20260811 10:07:35.877525 30411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:35.879060 30411 server_base.cc:1034] running on GCE node
W20260811 10:07:35.879056 30528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.879127 30531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:35.879218 30529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:35.879386 30411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:35.879568 30411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:35.880767 30411 hybrid_clock.cc:648] HybridClock initialized: now 1786442855880754 us; error 30 us; skew 500 ppm
Aug 11 10:07:35 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:35.882364 30411 init.cc:377] Logged in from keytab as oryx/127.22.177.61@KRBTEST.COM (short username oryx)
I20260811 10:07:35.882665 30411 webserver.cc:460] Webserver started at http://127.22.177.61:34855/ using document root <none> and password file <none>
I20260811 10:07:35.882802 30411 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:35.882854 30411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:35.883533 30411 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:35.884224 30538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:35.884398 30411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:35.884461 30411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
uuid: "369c82f45c084c0ba7e646b970b38f1e"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "09723070c9ac66707e2c41a2a066c8ef"
server_key_iv: "4e0eebdd0c1e1142c24b09adb37018ae"
server_key_version: "encryptionkey@0"
I20260811 10:07:35.884521 30411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:35.890836 30411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:35.892619 30411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:35.895824 30411 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.61:44937
I20260811 10:07:35.896294 30411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
I20260811 10:07:35.896369 30593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.61:44937 every 8 connection(s)
I20260811 10:07:35.897024 30594 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:07:35.897830 30594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Bootstrap starting.
I20260811 10:07:35.900954 30594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:07:35.901206 30594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Bootstrap complete.
I20260811 10:07:35.901461 30594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } }
I20260811 10:07:35.901556 30594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 369c82f45c084c0ba7e646b970b38f1e, State: Initialized, Role: LEARNER
I20260811 10:07:35.901635 30594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } }
I20260811 10:07:35.901834 30594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:35.901831 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } }
I20260811 10:07:35.901930 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: LEARNER
I20260811 10:07:35.904080 30610 catalog_manager.cc:1259] Loaded cluster ID: 1dc09f59a80048908d229b2985f3ca62
I20260811 10:07:35.904135 30610 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: loading cluster ID for follower catalog manager: success
I20260811 10:07:35.905229 30610 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: acquiring CA information for follower catalog manager: success
I20260811 10:07:35.905539 30610 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:07:35.906056 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:35.897623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.22.177.61:44937)
0811 10:07:35.897800 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:35.897802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:35.897811 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:35.897834 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:35.897835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:35.897846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:35.898701 (+ 855us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:35.898739 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.899697 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.900732 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:35.901055 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:35.901174 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:35.902737 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:35.902739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:35.902739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:35.902754 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:35.904541 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.904908 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.904910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.904910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.904974 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:35.905187 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:35.905188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:35.905188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:35.905340 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:35.905441 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:35.905891 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 10:07:35.905994 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for oryx/127.22.177.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.60.keytab.
Entry for principal oryx/127.22.177.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.60.keytab.
I20260811 10:07:35.942065 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:40081
--webserver_interface=127.22.177.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.60.keytab
--principal=oryx/127.22.177.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:36.047153 30614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:36.047484 30614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:36.047520 30614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:36.049145 30614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:36.049208 30614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:36.049223 30614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:36.049237 30614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:36.049249 30614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:36.050961 30614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.60.keytab
--principal=oryx/127.22.177.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:40081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.177.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:36.051186 30614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:36.051398 30614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:36.057041 30624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.057050 30621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.057050 30622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:36.057541 30614 server_base.cc:1034] running on GCE node
I20260811 10:07:36.057848 30614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:36.058176 30614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:36.059361 30614 hybrid_clock.cc:648] HybridClock initialized: now 1786442856059346 us; error 27 us; skew 500 ppm
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:36.062146 30614 init.cc:377] Logged in from keytab as oryx/127.22.177.60@KRBTEST.COM (short username oryx)
I20260811 10:07:36.062676 30614 webserver.cc:460] Webserver started at http://127.22.177.60:43129/ using document root <none> and password file <none>
I20260811 10:07:36.062933 30614 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:36.063004 30614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:36.063156 30614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:36.064271 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/instance:
uuid: "32f892a7e1e442669aeff9c58f79e3d9"
format_stamp: "Formatted at 2026-08-11 10:07:36 on dist-test-slave-g350"
server_key: "689477f4cb944bb33f241378ed0d632d"
server_key_iv: "470f11b7971d7833725e281105ecf582"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.064690 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal/instance:
uuid: "32f892a7e1e442669aeff9c58f79e3d9"
format_stamp: "Formatted at 2026-08-11 10:07:36 on dist-test-slave-g350"
server_key: "689477f4cb944bb33f241378ed0d632d"
server_key_iv: "470f11b7971d7833725e281105ecf582"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.066205 30614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:36.067225 30631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.067480 30614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:36.067600 30614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
uuid: "32f892a7e1e442669aeff9c58f79e3d9"
format_stamp: "Formatted at 2026-08-11 10:07:36 on dist-test-slave-g350"
server_key: "689477f4cb944bb33f241378ed0d632d"
server_key_iv: "470f11b7971d7833725e281105ecf582"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.067679 30614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:36.099592 30614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:36.102154 30614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:36.102363 30614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:36.107235 30614 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.60:40081
I20260811 10:07:36.107309 30689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.60:40081 every 8 connection(s)
I20260811 10:07:36.107702 30614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
I20260811 10:07:36.108853 30691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:36.110684 30691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:07:36.111100 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 30614
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.60@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.177.60@KRBTEST.COM
I20260811 10:07:36.122102 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.111749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.22.177.62:37093)
0811 10:07:36.111802 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.111804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.112104 (+ 300us) server_negotiation.cc:407] Connection header received
0811 10:07:36.112684 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.112686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.112700 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.112789 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.113471 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.114025 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.115201 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.115461 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.115512 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.118409 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.118410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.118410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.118417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.120382 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.120822 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.120824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.120824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.120888 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.121196 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.121197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.121197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.121326 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.121390 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.121906 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.121997 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:07:36.122308 30694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.111674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:57216)
0811 10:07:36.111951 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.112000 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.112541 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.112836 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.112839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.113143 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.113366 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.113372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.114135 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.114136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.115113 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.115116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.115559 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.115560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.115685 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.116398 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.116409 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.118275 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.120476 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.120483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.120485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.120724 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.120968 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.120968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.120969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.121113 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.121410 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.121418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.121675 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.121875 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.122042 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":186,"threads_started":1}
I20260811 10:07:36.122680 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.111689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.22.177.60:40081)
0811 10:07:36.111930 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.111942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.111978 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:07:36.112538 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.112549 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.112577 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.112678 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.113352 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.114583 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.115350 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.115768 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.115847 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.119054 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.119064 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.119064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.119075 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.121045 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.121560 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.121563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.121563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.121663 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.121887 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.121888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.121888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.122066 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.122438 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.122499 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.122573 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":151,"threads_started":1}
I20260811 10:07:36.123047 30691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.60@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
I20260811 10:07:36.130481 30694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.123439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:53780)
0811 10:07:36.123503 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.123517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.123564 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.123714 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.123715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.123869 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.123910 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.123912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.124659 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.124661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.125217 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.125219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.125682 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.125683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.125811 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.126312 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.126317 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.127722 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.129250 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.129251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.129252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.129442 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.129719 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.129720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.129720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.129789 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.130092 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.130094 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.130181 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.130264 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.130374 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:07:36.130506 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.123548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.22.177.61:44937)
0811 10:07:36.123596 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.123598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.123607 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:36.123634 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.123636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.123646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.123710 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.124006 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.124564 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.125331 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.125564 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.125671 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.127828 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.127830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.127830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.127839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.129151 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.129528 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.129530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.129530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.129658 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.129875 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.129876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.129877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.130009 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.130090 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.130315 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.130411 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:07:36.131038 30691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.60@KRBTEST.COM for oryx/127.22.177.60@KRBTEST.COM
I20260811 10:07:36.138051 30694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.131412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:54022)
0811 10:07:36.131468 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.131480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.131531 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.131646 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.131647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.131768 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.131831 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.131832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.132425 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.132425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.132948 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.132949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.133434 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.133435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.133475 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.134036 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.134040 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.135323 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.137031 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.137032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.137032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.137206 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.137408 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.137409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.137409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.137466 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.137742 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.137743 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.137781 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.137860 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.137956 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 10:07:36.138131 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.131442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.22.177.60:40081)
0811 10:07:36.131499 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.131501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.131519 (+ 18us) server_negotiation.cc:407] Connection header received
0811 10:07:36.131588 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.131589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.131596 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.131645 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.131890 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.132355 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.133103 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.133357 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.133421 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.135406 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.135408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.135409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.135415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.136948 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.137284 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.137285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.137285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.137338 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.137525 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.137525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.137525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.137682 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.137756 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.137878 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.138004 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":37}
I20260811 10:07:36.139622 30691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Bootstrap starting.
I20260811 10:07:36.140547 30691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:36.140894 30691 log.cc:824] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:36.141846 30691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: No bootstrap required, opened a new log
I20260811 10:07:36.143396 30691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } }
I20260811 10:07:36.143534 30691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:36.143590 30691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32f892a7e1e442669aeff9c58f79e3d9, State: Initialized, Role: FOLLOWER
I20260811 10:07:36.143734 30691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } }
I20260811 10:07:36.144008 30696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } } }
I20260811 10:07:36.144047 30691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:36.144119 30696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:07:36.145884 30708 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:36.145946 30708 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:36.148488 30614 master_runner.cc:428] Detected that this master 32f892a7e1e442669aeff9c58f79e3d9 is joining an existing cluster
I20260811 10:07:36.148543 30614 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.177.60:40081
I20260811 10:07:36.157575 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.149725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.22.177.62:37093)
0811 10:07:36.149783 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.149785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.150128 (+ 343us) server_negotiation.cc:407] Connection header received
0811 10:07:36.150232 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.150234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.150246 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.150305 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.150782 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.151573 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.152784 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.153082 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.153123 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.154644 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.154646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.154646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.154654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.156415 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.156811 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.156811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.156812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.156865 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.157067 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.157068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.157068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.157178 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.157288 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.157445 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.157509 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 10:07:36.157671 30716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.149633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:57218)
0811 10:07:36.150060 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.150071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.150158 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.150319 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.150320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.150607 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.150680 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.150682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.151678 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.151679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.152677 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.152680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.153223 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.153223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.153289 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.153931 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.153937 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.154520 (+ 583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.156496 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.156497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.156498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.156735 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.156931 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.156932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.156933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.157004 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.157286 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.157287 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.157342 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.157423 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.157529 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":45,"thread_start_us":122,"threads_started":1}
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442855, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.62@KRBTEST.COM for oryx/127.22.177.60@KRBTEST.COM
I20260811 10:07:36.169129 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.159871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:54036)
0811 10:07:36.159927 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.159938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.159986 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.160286 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.160288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.160442 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.160483 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.160484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.161146 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.161147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.161715 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.161717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.162094 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.162095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.162162 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.162584 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.162589 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.164045 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.167952 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.167954 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.167954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.168210 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.168457 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.168457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.168458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.168516 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.168791 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.168792 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.168839 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.168902 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.169020 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 10:07:36.169575 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.159951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.22.177.60:40081)
0811 10:07:36.160128 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.160130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.160136 (+ 6us) server_negotiation.cc:407] Connection header received
0811 10:07:36.160165 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.160165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.160174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.160283 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.160567 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.161063 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.161796 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.162020 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.162096 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.164160 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.164162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.164163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.164170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.167836 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.168317 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.168318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.168318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.168382 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.168585 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.168585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.168585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.168715 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.169271 (+ 556us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.169315 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.169471 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260811 10:07:36.170118 29912 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.177.60:40081: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9"
member_type: NON_VOTER
last_known_addr {
host: "127.22.177.60"
port: 40081
}
attrs {
promote: true
}
}
dest_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a"
cas_config_opid_index: 5
I20260811 10:07:36.170426 29912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } }
I20260811 10:07:36.170625 30715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.149923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:54034)
0811 10:07:36.150301 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.150311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.150373 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.150523 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.150524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.150793 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.150867 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.150869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.151775 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.151776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.152691 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.152694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.153177 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.153178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.153243 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.166552 (+ 13309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.166558 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.166979 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.169228 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.169228 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.169228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.169496 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.169712 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.169713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.169713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.169978 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.170326 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.170327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.170359 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.170414 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.170512 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":5,"thread_start_us":78,"threads_started":1}
I20260811 10:07:36.170879 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.149952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.22.177.60:40081)
0811 10:07:36.149994 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.149996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.150353 (+ 357us) server_negotiation.cc:407] Connection header received
0811 10:07:36.150434 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.150435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.150443 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.150470 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.150952 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.151682 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.152786 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.153053 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.153091 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.167051 (+ 13960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.167052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.167052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.167059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.169139 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.169562 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.169563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.169563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.169642 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.170106 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.170107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.170107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.170247 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.170305 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.170736 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.170810 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:07:36.170850 30513 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.171526 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.149889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.22.177.61:44937)
0811 10:07:36.149941 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.149943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.150132 (+ 189us) server_negotiation.cc:407] Connection header received
0811 10:07:36.150356 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.150358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.150370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.150421 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.150962 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.151683 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.152802 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.153095 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.153135 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.166745 (+ 13610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.166748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.166748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.166757 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.168544 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.169054 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.169056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.169057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.169119 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.170153 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.170154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.170155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.170416 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.170493 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.171327 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.171418 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:07:36.171993 30513 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.177.60:40081
I20260811 10:07:36.171999 30663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:36.172084 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.172137 30714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.149635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:53788)
0811 10:07:36.150061 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.150074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.150146 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.150465 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.150465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.150797 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.150867 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.150869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.151784 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.151784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.152691 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.152694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.153204 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.153205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.153258 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.166063 (+ 12805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.166068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.166596 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.168646 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.168647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.168648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.168956 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.169585 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.169585 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.169586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.169920 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.170636 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.170637 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.171197 (+ 560us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.171809 (+ 612us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.171932 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":59,"spinlock_wait_cycles":9472,"thread_start_us":127,"threads_started":1}
I20260811 10:07:36.172168 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.172981 30663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Refusing update from remote peer 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 10:07:36.173372 30614 master.cc:537] Master@127.22.177.60:40081 shutting down...
I20260811 10:07:36.173432 30514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 10:07:36.174420 29896 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a -> Peer 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60:40081): Couldn't send request to peer 32f892a7e1e442669aeff9c58f79e3d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 10:07:36.178714 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.171341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:53796)
0811 10:07:36.174880 (+ 3539us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.174892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.174944 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.175273 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.175275 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.175276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.175322 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.175324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.176311 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.176312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.177569 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.177571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.178403 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.178404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.178487 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.178518 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.178574 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.178622 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3519}
I20260811 10:07:36.178755 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.171456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.22.177.61:44937)
0811 10:07:36.171792 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.171794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.175001 (+ 3207us) server_negotiation.cc:407] Connection header received
0811 10:07:36.175027 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.175027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.175051 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.175105 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.175402 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.176195 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.177697 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.178297 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.178371 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.178447 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.178632 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.178684 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":87,"threads_started":1}
I20260811 10:07:36.179006 30568 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 LEARNER]: Refusing update from remote peer 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 10:07:36.179292 30514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 10:07:36.179605 30614 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:36.179714 30614 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:36.179765 30614 tablet_replica.cc:330] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: stopping tablet replica
I20260811 10:07:36.179929 30568 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
W20260811 10:07:36.180878 30632 connection.cc:441] server connection from 127.0.0.1:54010 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:54010 (request call id 41) could send its response
I20260811 10:07:36.181283 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f5a9fbb091449f0bf7ff897bd7d6c8a. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.181386 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: LEARNER
I20260811 10:07:36.181517 30721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: NON_VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: true } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.181722 30721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: LEARNER
I20260811 10:07:36.182260 30514 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: attempting to promote NON_VOTER 369c82f45c084c0ba7e646b970b38f1e to VOTER
I20260811 10:07:36.182473 30514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } }
I20260811 10:07:36.182991 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.182850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:54044)
0811 10:07:36.182917 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.182961 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:40081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 10:07:36.183027 30568 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 LEARNER]: Refusing update from remote peer 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 10:07:36.183393 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.183200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:54052)
0811 10:07:36.183339 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.183368 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:40081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 10:07:36.183399 30722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 10:07:36.183543 29896 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a -> Peer 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60:40081): Couldn't send request to peer 32f892a7e1e442669aeff9c58f79e3d9. Status: Network error: Client connection negotiation failed: client connection to 127.22.177.60:40081: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 10:07:36.184121 30514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 369c82f45c084c0ba7e646b970b38f1e (127.22.177.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.184326 30568 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 369c82f45c084c0ba7e646b970b38f1e (127.22.177.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.185076 30514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.185158 30514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.185110 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.185217 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.185405 30721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.185480 30721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:07:36.193140 30614 master.cc:552] Master@127.22.177.60:40081 shutdown complete.
I20260811 10:07:36.193198 30614 master_runner.cc:305] Clearing existing system tablet
I20260811 10:07:36.193336 30614 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:36.193387 30614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:36.194159 30614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:36.194700 30724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.194893 30614 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:36.194958 30614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:36.194984 30614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
uuid: "32f892a7e1e442669aeff9c58f79e3d9"
format_stamp: "Formatted at 2026-08-11 10:07:36 on dist-test-slave-g350"
server_key: "689477f4cb944bb33f241378ed0d632d"
server_key_iv: "470f11b7971d7833725e281105ecf582"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.195869 30614 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:36.197681 30614 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 10:07:36.197758 30614 log.cc:1196] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 10:07:36.197957 30614 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Deleting consensus metadata
I20260811 10:07:36.198086 30614 master_runner.cc:315] Copying system tablet from 127.22.177.62:37093
I20260811 10:07:36.198745 30614 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: tablet copy: Beginning tablet copy session from remote peer at address 127.22.177.62:37093
I20260811 10:07:36.206033 30730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.199113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:57220)
0811 10:07:36.199326 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.199338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.199420 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.199569 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.199569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.199754 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.199817 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.199819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.200784 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.200785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.201387 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.201390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.201917 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.201918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.201989 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.202569 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.202574 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.202972 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.204838 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.204839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.204839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.205014 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.205263 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.205264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.205264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.205331 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.205720 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.205721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.205764 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.205832 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.205924 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":172,"threads_started":1}
I20260811 10:07:36.206055 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.199156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.22.177.62:37093)
0811 10:07:36.199213 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.199216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.199426 (+ 210us) server_negotiation.cc:407] Connection header received
0811 10:07:36.199464 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.199465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.199478 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.199557 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.199894 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.200687 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.201481 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.201833 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.201908 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.203055 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.203057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.203057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.203066 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.204767 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.205089 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.205090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.205091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.205157 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.205408 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.205409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.205410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.205549 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.205728 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.205879 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.205960 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:07:36.206643 29936 tablet_copy_service.cc:140] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 32f892a7e1e442669aeff9c58f79e3d9 ({username='oryx', principal='oryx/127.22.177.60@KRBTEST.COM'} at 127.0.0.1:57220)
I20260811 10:07:36.206717 29936 tablet_copy_service.cc:161] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 32f892a7e1e442669aeff9c58f79e3d9 at {username='oryx', principal='oryx/127.22.177.60@KRBTEST.COM'} at 127.0.0.1:57220: session id = 32f892a7e1e442669aeff9c58f79e3d9-00000000000000000000000000000000
I20260811 10:07:36.207324 29936 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 10:07:36.207909 30614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 10:07:36.209337 30614 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: tablet copy: Starting download of 0 data blocks...
I20260811 10:07:36.209496 30614 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: tablet copy: Starting download of 1 WAL segments...
I20260811 10:07:36.210413 30614 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 10:07:36.211705 29936 tablet_copy_service.cc:342] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Request end of tablet copy session 32f892a7e1e442669aeff9c58f79e3d9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.177.60@KRBTEST.COM'} at 127.0.0.1:57220
I20260811 10:07:36.211802 29936 tablet_copy_service.cc:433] P 3f5a9fbb091449f0bf7ff897bd7d6c8a: ending tablet copy session 32f892a7e1e442669aeff9c58f79e3d9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 32f892a7e1e442669aeff9c58f79e3d9
I20260811 10:07:36.215143 30614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:36.216491 30732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.216492 30731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.216661 30734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:36.216876 30614 server_base.cc:1034] running on GCE node
I20260811 10:07:36.216974 30614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:36.217136 30614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:36.218269 30614 hybrid_clock.cc:648] HybridClock initialized: now 1786442856218250 us; error 36 us; skew 500 ppm
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:36.219945 30614 init.cc:377] Logged in from keytab as oryx/127.22.177.60@KRBTEST.COM (short username oryx)
I20260811 10:07:36.220311 30614 webserver.cc:460] Webserver started at http://127.22.177.60:34323/ using document root <none> and password file <none>
I20260811 10:07:36.220467 30614 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:36.220515 30614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:36.221251 30614 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:36.221931 30741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.222074 30614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.222128 30614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
uuid: "32f892a7e1e442669aeff9c58f79e3d9"
format_stamp: "Formatted at 2026-08-11 10:07:36 on dist-test-slave-g350"
server_key: "689477f4cb944bb33f241378ed0d632d"
server_key_iv: "470f11b7971d7833725e281105ecf582"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.222199 30614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:36.227818 30614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:36.229547 30614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:36.233307 30614 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.60:40081
I20260811 10:07:36.233894 30614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
I20260811 10:07:36.233953 30799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.60:40081 every 8 connection(s)
I20260811 10:07:36.234858 30800 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:07:36.235423 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Bootstrap starting.
I20260811 10:07:36.239099 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:07:36.239389 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: Bootstrap complete.
I20260811 10:07:36.239634 30800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } }
I20260811 10:07:36.239740 30800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 32f892a7e1e442669aeff9c58f79e3d9, State: Initialized, Role: LEARNER
I20260811 10:07:36.239784 30800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } }
I20260811 10:07:36.239985 30800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:36.240007 30803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.240160 30803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 10:07:36.241709 30814 catalog_manager.cc:1259] Loaded cluster ID: 1dc09f59a80048908d229b2985f3ca62
I20260811 10:07:36.241766 30814 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: loading cluster ID for follower catalog manager: success
I20260811 10:07:36.242931 30814 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: acquiring CA information for follower catalog manager: success
I20260811 10:07:36.243235 30814 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:07:36.272838 30120 heartbeater.cc:503] Master 127.22.177.62:37093 was elected leader, sending a full tablet report...
I20260811 10:07:36.290671 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.282407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.22.177.60:40081)
0811 10:07:36.282580 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.282582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.282590 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:36.282613 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.282614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.282623 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.282683 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.283492 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.284423 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.285500 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.285898 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.285959 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.287557 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.287558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.287558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.287565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.289250 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.289705 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.289706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.289706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.289766 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.290012 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.290014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.290014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.290188 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.290309 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.290433 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.290506 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260811 10:07:36.291375 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29981
I20260811 10:07:36.307307 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:40717
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:37093
--webserver_port=36185
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:36.395671 30816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:36.395920 30816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:36.395947 30816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:36.397125 30816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:36.397166 30816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:36.397194 30816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:36.398705 30816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:40717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=36185
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:36.398912 30816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:36.399102 30816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:36.405058 30824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.405058 30822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.405068 30821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:36.405670 30816 server_base.cc:1034] running on GCE node
I20260811 10:07:36.405915 30816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:36.406282 30816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:36.407472 30816 hybrid_clock.cc:648] HybridClock initialized: now 1786442856407459 us; error 28 us; skew 500 ppm
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:36.410501 30816 init.cc:377] Logged in from keytab as oryx/127.22.177.1@KRBTEST.COM (short username oryx)
I20260811 10:07:36.411034 30816 webserver.cc:460] Webserver started at http://127.22.177.1:36185/ using document root <none> and password file <none>
I20260811 10:07:36.411309 30816 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:36.411403 30816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:36.412936 30816 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:36.413872 30831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.414112 30816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:36.414230 30816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "ae22e677e6f9c5704c4eedd52aa3b69e"
server_key_iv: "ff3eaf8d7b45baf6cb8b80177065746e"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.414719 30816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:36.427119 30816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:36.429679 30816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:36.429855 30816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:36.430099 30816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:36.430424 30816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:36.430478 30816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.430521 30816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:36.430549 30816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.436712 30816 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:40717
I20260811 10:07:36.436792 30956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:40717 every 8 connection(s)
I20260811 10:07:36.437163 30816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:36.441455 30264 heartbeater.cc:503] Master 127.22.177.62:37093 was elected leader, sending a full tablet report...
I20260811 10:07:36.443972 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 30816
I20260811 10:07:36.444152 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 30125
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.60@KRBTEST.COM
I20260811 10:07:36.452395 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.438222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:44641 (local address 127.22.177.62:37093)
0811 10:07:36.438272 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.438273 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.438647 (+ 374us) server_negotiation.cc:407] Connection header received
0811 10:07:36.439529 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.439530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.439546 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.439579 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.440441 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.441319 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442618 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.442851 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442922 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.448731 (+ 5809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.448734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.448734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.448743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.450824 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.451316 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.451317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.451318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.451372 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.451621 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.451622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.451622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.451774 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.451815 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.452164 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.452262 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:07:36.453258 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.437944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.1:44641)
0811 10:07:36.438577 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.438584 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.439414 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.439671 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.439672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.440130 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.440327 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.440333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.441650 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.441651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.442499 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.442501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.442936 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.443015 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.443716 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.443723 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.448600 (+ 4877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.450916 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.450925 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.450928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.451235 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.451438 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.451439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.451439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.451561 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.451858 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.451865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.452026 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.452783 (+ 757us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.452981 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":79,"thread_start_us":93,"threads_started":1}
I20260811 10:07:36.454106 30958 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37093
I20260811 10:07:36.454209 30958 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.454388 30958 heartbeater.cc:511] Master 127.22.177.62:37093 requested a full tablet report, sending...
I20260811 10:07:36.454850 29912 ts_manager.cc:194] Re-registered known tserver with Master: ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717)
I20260811 10:07:36.455422 29912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:44641
I20260811 10:07:36.456995 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.437944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.22.177.1:54841)
0811 10:07:36.438428 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.438438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.439414 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.439671 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.439672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.440124 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.440327 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.440333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.441324 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.441325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.442435 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.442438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.442994 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.443047 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.443709 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.443765 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.448978 (+ 5213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.455810 (+ 6832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.455811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.455811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.456112 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.456301 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.456302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.456302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.456362 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.456637 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.456638 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.456681 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.456766 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.456885 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":22,"thread_start_us":125,"threads_started":1}
I20260811 10:07:36.457003 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.438189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:54841 (local address 127.22.177.60:40081)
0811 10:07:36.438249 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.438252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.438538 (+ 286us) server_negotiation.cc:407] Connection header received
0811 10:07:36.439530 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.439531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.439546 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.439579 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.440466 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.441220 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442548 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.442880 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442945 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.449094 (+ 6149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.449095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.449096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.449116 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.455737 (+ 6621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.456176 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.456178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.456178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.456246 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.456442 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.456443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.456443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.456562 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.456633 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.456808 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.456916 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:07:36.457346 30957 heartbeater.cc:347] Connected to a master server at 127.22.177.60:40081
I20260811 10:07:36.457402 30957 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.457459 30957 heartbeater.cc:511] Master 127.22.177.60:40081 requested a full tablet report, sending...
I20260811 10:07:36.457858 30758 ts_manager.cc:194] Registered new tserver with Master: ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717)
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
I20260811 10:07:36.460738 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:41947
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:37093
--webserver_port=36449
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:07:36.462904 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.438128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:39571 (local address 127.22.177.61:44937)
0811 10:07:36.438218 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.438220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.438482 (+ 262us) server_negotiation.cc:407] Connection header received
0811 10:07:36.439499 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.439501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.439513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.439569 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.440430 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.441090 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442477 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.442759 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442827 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.459659 (+ 16832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.459662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.459662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.459674 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.461673 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.462016 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.462017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.462018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.462084 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.462312 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.462314 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.462314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.462493 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.462571 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.462748 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.462828 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 10:07:36.463011 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.437943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.22.177.1:39571)
0811 10:07:36.438354 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.438388 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.439387 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.439634 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.439637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.440101 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.440325 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.440329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.441208 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.441209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.442386 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.442389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.442844 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.442845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.443014 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.443805 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.443811 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.459547 (+ 15736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.461770 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.461771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.461772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.461943 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.462166 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.462167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.462167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.462225 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.462575 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.462577 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.462635 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.462717 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.462856 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":40,"thread_start_us":119,"threads_started":1}
I20260811 10:07:36.463500 30959 heartbeater.cc:347] Connected to a master server at 127.22.177.61:44937
I20260811 10:07:36.463575 30959 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.463649 30959 heartbeater.cc:511] Master 127.22.177.61:44937 requested a full tablet report, sending...
I20260811 10:07:36.464040 30557 ts_manager.cc:194] Registered new tserver with Master: ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717)
W20260811 10:07:36.548362 30964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:36.548617 30964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:36.548661 30964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:36.549898 30964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:36.549969 30964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:36.550016 30964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:36.551610 30964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:41947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=36449
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:36.551892 30964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:36.552134 30964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:36.558629 30969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.558658 30970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.558748 30972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:36.559051 30964 server_base.cc:1034] running on GCE node
I20260811 10:07:36.559357 30964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:36.559629 30964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:36.560784 30964 hybrid_clock.cc:648] HybridClock initialized: now 1786442856560771 us; error 32 us; skew 500 ppm
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:36.563649 30964 init.cc:377] Logged in from keytab as oryx/127.22.177.2@KRBTEST.COM (short username oryx)
I20260811 10:07:36.564191 30964 webserver.cc:460] Webserver started at http://127.22.177.2:36449/ using document root <none> and password file <none>
I20260811 10:07:36.564502 30964 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:36.564589 30964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:36.566100 30964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:36.566852 30980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.567057 30964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:36.567164 30964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "8b1262c1389b4425a6a42cb6df58fae6"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "f343667d7cfb79cfa91c1ccb923e5505"
server_key_iv: "bc7ace6e7717d644b6c2621679ffd65e"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.567605 30964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:36.586016 30964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:36.588836 30964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:36.589037 30964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:36.589342 30964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:36.589756 30964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:36.589823 30964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.589862 30964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:36.589892 30964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.596845 30964 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:41947
I20260811 10:07:36.596897 31102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:41947 every 8 connection(s)
I20260811 10:07:36.597293 30964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:36.597831 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 30964
I20260811 10:07:36.598026 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 30269
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
I20260811 10:07:36.609154 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:41367
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:37093
--webserver_port=45105
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.60@KRBTEST.COM
I20260811 10:07:36.612709 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.598203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:32903)
0811 10:07:36.599274 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.599329 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.599904 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.600195 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.600200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.600996 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.601212 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.601219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.602277 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.602279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.603855 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.603857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.604438 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.604439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.604590 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.605240 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.605252 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.607340 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.609578 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.609587 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.609613 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.610068 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.611122 (+ 1054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.611123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.611123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.611292 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.611666 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.611672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.611893 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.612037 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.612260 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":84,"threads_started":1}
I20260811 10:07:36.613297 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.598474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:32903 (local address 127.22.177.62:37093)
0811 10:07:36.598525 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.598527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.599425 (+ 898us) server_negotiation.cc:407] Connection header received
0811 10:07:36.600029 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.600031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.600045 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.600118 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.601321 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.602138 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.603952 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.604296 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.604341 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.607461 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.607463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.607463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.607473 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.609487 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.610178 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.610180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.610180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.611035 (+ 855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.611399 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.611400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.611401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.611579 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.611654 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.613051 (+ 1397us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.613159 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:07:36.613941 31109 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37093
I20260811 10:07:36.614035 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.599277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:57371 (local address 127.22.177.60:40081)
0811 10:07:36.599329 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.599332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.603216 (+ 3884us) server_negotiation.cc:407] Connection header received
0811 10:07:36.603255 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.603257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.603281 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.603353 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.603810 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.604605 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.606323 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.606673 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.606735 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.610366 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.610368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.610368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.610378 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.612508 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.613017 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.613018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.613019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.613090 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.613337 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.613339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.613339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.613501 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.613578 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.613839 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.613929 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:07:36.614085 31109 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.614104 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.598204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.22.177.2:57371)
0811 10:07:36.603119 (+ 4915us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.603132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.603197 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.603368 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.603369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.603591 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.603699 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.603701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.604724 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.604724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.606203 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.606207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.606969 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.606970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.607039 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.608093 (+ 1054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.608099 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.610241 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.612612 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.612613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.612613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.612908 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.613173 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.613174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.613174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.613236 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.613639 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.613640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.613695 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.613788 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.613912 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4885,"mutex_wait_us":92,"thread_start_us":160,"threads_started":1}
I20260811 10:07:36.614308 31109 heartbeater.cc:511] Master 127.22.177.62:37093 requested a full tablet report, sending...
I20260811 10:07:36.614816 29912 ts_manager.cc:194] Re-registered known tserver with Master: 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947)
I20260811 10:07:36.614975 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.599372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:49957 (local address 127.22.177.61:44937)
0811 10:07:36.599434 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.599437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.603516 (+ 4079us) server_negotiation.cc:407] Connection header received
0811 10:07:36.603623 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.603624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.603638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.603670 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.604146 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.605012 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.606446 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.606815 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.606861 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.610148 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.610150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.610150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.610160 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.612684 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.613152 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.613153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.613154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.613236 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.614222 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.614223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.614224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.614382 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.614445 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.614794 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.614891 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:07:36.615332 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.598582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.22.177.2:49957)
0811 10:07:36.603443 (+ 4861us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.603455 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.603547 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.603741 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.603742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.603962 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.604041 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.604043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.605130 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.605131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.606331 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.606334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.607044 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.607045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.607126 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.607664 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.607670 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.610021 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.612793 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.612795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.612795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.613055 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.613336 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.613336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.613337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.613407 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.614600 (+ 1193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.614602 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.614651 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.615099 (+ 448us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.615222 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4840,"mutex_wait_us":33,"thread_start_us":607,"threads_started":1}
I20260811 10:07:36.615386 29912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:32903
I20260811 10:07:36.615679 31110 heartbeater.cc:347] Connected to a master server at 127.22.177.61:44937
I20260811 10:07:36.615736 31110 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.615805 31110 heartbeater.cc:511] Master 127.22.177.61:44937 requested a full tablet report, sending...
I20260811 10:07:36.616133 30557 ts_manager.cc:194] Registered new tserver with Master: 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947)
I20260811 10:07:36.614705 31105 heartbeater.cc:347] Connected to a master server at 127.22.177.60:40081
I20260811 10:07:36.616379 31105 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.616456 31105 heartbeater.cc:511] Master 127.22.177.60:40081 requested a full tablet report, sending...
I20260811 10:07:36.616760 30758 ts_manager.cc:194] Registered new tserver with Master: 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947)
W20260811 10:07:36.710178 31116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:36.710454 31116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:36.710502 31116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:36.711869 31116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:36.711947 31116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:36.711999 31116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:36.713804 31116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:41367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=45105
--tserver_master_addrs=127.22.177.62:37093,127.22.177.61:44937,127.22.177.60:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:36.714123 31116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:36.714396 31116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:36.720731 31116 server_base.cc:1034] running on GCE node
W20260811 10:07:36.720692 31121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.720692 31122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:36.720692 31124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:36.721179 31116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:36.721455 31116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:36.722648 31116 hybrid_clock.cc:648] HybridClock initialized: now 1786442856722624 us; error 93 us; skew 500 ppm
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:36.725723 31116 init.cc:377] Logged in from keytab as oryx/127.22.177.3@KRBTEST.COM (short username oryx)
I20260811 10:07:36.726269 31116 webserver.cc:460] Webserver started at http://127.22.177.3:45105/ using document root <none> and password file <none>
I20260811 10:07:36.726521 31116 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:36.726584 31116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:36.728123 31116 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:07:36.728677 31131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.728870 31116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:36.728986 31116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "de5bedd438b448b9992a005cdb498bb9"
format_stamp: "Formatted at 2026-08-11 10:07:35 on dist-test-slave-g350"
server_key: "dd55db31987e86a760d79c0960bfc440"
server_key_iv: "54013d0fd64ef4d689b9d9cfaf74934c"
server_key_version: "encryptionkey@0"
I20260811 10:07:36.729375 31116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:36.740444 31116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:36.743012 31116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:36.743171 31116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:36.743427 31116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:36.743757 31116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:36.743790 31116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.743814 31116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:36.743857 31116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:36.749974 31116 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:41367
I20260811 10:07:36.750061 31251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:41367 every 8 connection(s)
I20260811 10:07:36.750414 31116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.60@KRBTEST.COM
I20260811 10:07:36.758709 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 31116
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
Aug 11 10:07:36 dist-test-slave-g350 krb5kdc[29862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442856, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.61@KRBTEST.COM
I20260811 10:07:36.763816 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.751206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:50391 (local address 127.22.177.60:40081)
0811 10:07:36.751536 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.751538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.751891 (+ 353us) server_negotiation.cc:407] Connection header received
0811 10:07:36.752903 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.752905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.752917 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.752951 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.753700 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.754469 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.755693 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.756009 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.756092 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.760207 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.760208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.760209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.760217 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.762042 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.762582 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.762584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.762584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.762653 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.762969 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.762970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.762970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.763121 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.763173 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.763593 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.763685 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305}
I20260811 10:07:36.764391 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.751219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.22.177.3:50391)
0811 10:07:36.751835 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.751844 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.752796 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.753041 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.753042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.753362 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.753541 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.753548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.754545 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.754545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.755566 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.755569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.756164 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.756166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.756316 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.757028 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.757037 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.760094 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.762137 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.762144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.762146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.762492 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.762742 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.762743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.762743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.762889 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.763228 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.763235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.763418 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.763920 (+ 502us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.764118 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":107,"thread_start_us":169,"threads_started":1}
I20260811 10:07:36.765077 31252 heartbeater.cc:347] Connected to a master server at 127.22.177.60:40081
I20260811 10:07:36.765190 31252 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.765410 31252 heartbeater.cc:511] Master 127.22.177.60:40081 requested a full tablet report, sending...
I20260811 10:07:36.765892 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.751228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:57333 (local address 127.22.177.62:37093)
0811 10:07:36.751286 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.751288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.751748 (+ 460us) server_negotiation.cc:407] Connection header received
0811 10:07:36.752865 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.752866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.752880 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.752939 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.753659 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.754461 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.755819 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.756125 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.756189 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.759928 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.759930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.759930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.759940 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.762472 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.764441 (+ 1969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.764443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.764443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.764509 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.765299 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.765300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.765301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.765444 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.765511 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.765724 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.765805 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:07:36.765949 30758 ts_manager.cc:194] Registered new tserver with Master: de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367)
I20260811 10:07:36.766034 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.751220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:57333)
0811 10:07:36.751615 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.751661 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.752755 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.753014 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.753017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.753338 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.753539 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.753543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.754655 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.754656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.755701 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.755704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.756201 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.756202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.756319 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.757066 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.757072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.759785 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.762566 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.762567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.762568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.762777 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.764882 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.764883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.764883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.765202 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.765538 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.765539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.765579 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.765669 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.765774 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":59,"thread_start_us":171,"threads_started":1}
I20260811 10:07:36.766620 31253 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37093
I20260811 10:07:36.766685 31253 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.766772 31253 heartbeater.cc:511] Master 127.22.177.62:37093 requested a full tablet report, sending...
I20260811 10:07:36.767035 29912 ts_manager.cc:194] Re-registered known tserver with Master: de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367)
I20260811 10:07:36.767400 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.751710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.22.177.3:60835)
0811 10:07:36.751935 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.751942 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.752790 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.753041 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.753041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.753363 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:36.753541 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.753548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.754415 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.754415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.755706 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.755708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.756201 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.756202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.756319 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.757185 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:36.757190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:36.761055 (+ 3865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:36.764398 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.764399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.764399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.764797 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.766713 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:36.766714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:36.766714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:36.766781 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:36.767086 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:36.767087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:36.767125 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.767187 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.767299 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":96,"thread_start_us":78,"threads_started":1}
I20260811 10:07:36.767526 29912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:57333
I20260811 10:07:36.767757 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.751787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:60835 (local address 127.22.177.61:44937)
0811 10:07:36.751860 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.751863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.751990 (+ 127us) server_negotiation.cc:407] Connection header received
0811 10:07:36.752904 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.752905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.752918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.752951 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.753668 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.754299 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.755839 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.756065 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.756107 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.761164 (+ 5057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.761166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.761166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.761175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.764254 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.766385 (+ 2131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.766387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.766387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.766456 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.766856 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.766858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.766858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.766994 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.767057 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.767583 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.767670 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:07:36.768107 31254 heartbeater.cc:347] Connected to a master server at 127.22.177.61:44937
I20260811 10:07:36.768158 31254 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:36.768229 31254 heartbeater.cc:511] Master 127.22.177.61:44937 requested a full tablet report, sending...
I20260811 10:07:36.768522 30557 ts_manager.cc:194] Registered new tserver with Master: de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367)
I20260811 10:07:36.771613 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.760897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.22.177.61:44937)
0811 10:07:36.761050 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.761053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.761063 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:36.761087 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.761088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.761102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.761182 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.762204 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.763093 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.764241 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.764599 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.764680 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.768252 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.768254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.768254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.768305 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.770191 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.770617 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.770619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.770619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.770679 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.770910 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.770911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.770912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.771077 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.771185 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.771385 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.771477 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260811 10:07:36.772325 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.759928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.22.177.60:40081)
0811 10:07:36.761808 (+ 1880us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.761810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.761818 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:36.761843 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.761844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.761858 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.761924 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.762684 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.763477 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.765419 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.765814 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.765876 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.767764 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.767766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.767766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.767774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.770759 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.771115 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.771116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.771117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.771181 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.771457 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.771459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.771459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.771608 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.771711 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.772159 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.772254 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1853,"thread_start_us":60,"threads_started":1}
I20260811 10:07:36.775441 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.760367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.22.177.62:37093)
0811 10:07:36.760502 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.760504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.766613 (+ 6109us) server_negotiation.cc:407] Connection header received
0811 10:07:36.767999 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.768001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.768017 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.768063 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.768777 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.769693 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.771051 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.771369 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.771453 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.772941 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:36.772942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:36.772943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:36.772949 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:36.774396 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.774783 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.774783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.774784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.774836 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:36.775011 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:36.775012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:36.775012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:36.775102 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:36.775166 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.775305 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.775376 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260811 10:07:36.777356 29912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:36.779090 29912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:36.782982 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.778792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:54070)
0811 10:07:36.779479 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.779489 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.779577 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.779742 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.779743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.779745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.779798 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.779799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.780710 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.780712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.781879 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.781882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.782720 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.782721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.782786 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.782791 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.782848 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.782902 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":92,"threads_started":1}
I20260811 10:07:36.783033 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.778882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.22.177.60:40081)
0811 10:07:36.778936 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.778939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.779564 (+ 625us) server_negotiation.cc:407] Connection header received
0811 10:07:36.779637 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.779638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.779667 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.779735 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.779900 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.780594 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.781957 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.782632 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.782708 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.782791 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.782910 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.782958 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 10:07:36.783483 30770 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 10:07:36.783752 31276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f5a9fbb091449f0bf7ff897bd7d6c8a. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.784056 31276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 10:07:36.784122 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: NON_VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: true } } }
I20260811 10:07:36.784233 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.786139 31273 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: attempting to promote NON_VOTER 32f892a7e1e442669aeff9c58f79e3d9 to VOTER
I20260811 10:07:36.786315 31273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } }
I20260811 10:07:36.786906 30568 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Refusing update from remote peer 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260811 10:07:36.786978 30770 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 LEARNER]: Refusing update from remote peer 3f5a9fbb091449f0bf7ff897bd7d6c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260811 10:07:36.787195 31273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 10:07:36.787307 31273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 10:07:36.787853 30513 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.788348 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.783457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:40717 (local address 127.0.0.1:53158)
0811 10:07:36.784323 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.784333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.784392 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.784628 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.784629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.784631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.784678 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.784680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.785985 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.785986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.787159 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.787162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.788132 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.788133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.788206 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.788211 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.788241 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.788283 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761}
I20260811 10:07:36.788781 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.783816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.22.177.1:40717)
0811 10:07:36.784033 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.784039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.784452 (+ 413us) server_negotiation.cc:407] Connection header received
0811 10:07:36.784510 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.784512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.784536 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.784633 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.784737 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.785872 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.787249 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.788030 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.788504 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.788600 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.788628 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.788670 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260811 10:07:36.788929 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.788998 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.789050 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.789093 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.789176 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.789218 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:36.789484 30770 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.789521 30883 tablet_service.cc:1467] Processing CreateTablet for tablet 71de8356ea6f436fae0d65373e938232 (DEFAULT_TABLE table=test-table [id=bf32c9ddc0854d678222c3ae83f8b175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:36.789866 30568 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.789963 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71de8356ea6f436fae0d65373e938232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:36.790335 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.784116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:41947 (local address 127.0.0.1:56126)
0811 10:07:36.784457 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.784468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.784516 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.786160 (+ 1644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.786161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.786164 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.786202 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.786203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.787632 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.787633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.788771 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.788772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.790075 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.790076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.790137 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.790142 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.790172 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.790224 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":80,"threads_started":1}
I20260811 10:07:36.790771 31276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.790855 31276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:07:36.791159 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.784224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56126 (local address 127.22.177.2:41947)
0811 10:07:36.785572 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.785576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.785610 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:07:36.785650 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.785652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.785667 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.786341 (+ 674us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.786410 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.787537 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.789173 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.789964 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.790443 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.790533 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.790563 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.790609 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1309,"thread_start_us":94,"threads_started":1}
I20260811 10:07:36.791973 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.783569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41367 (local address 127.0.0.1:43872)
0811 10:07:36.784252 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.784263 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.784305 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.784850 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.784850 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.784852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.784889 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.784890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.787717 (+ 2827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.787718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.788768 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.788771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.791716 (+ 2945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.791718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.791777 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.791786 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.791843 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.791887 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":102,"threads_started":1}
I20260811 10:07:36.792239 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.783667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.22.177.3:41367)
0811 10:07:36.784671 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.784675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.784685 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:36.784735 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.784737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.784753 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.784840 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.784952 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.786014 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.788883 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.791599 (+ 2716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.791722 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.791802 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.791860 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.792125 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":93,"threads_started":1}
I20260811 10:07:36.793455 31289 tablet_bootstrap.cc:492] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Bootstrap starting.
I20260811 10:07:36.794188 31289 tablet_bootstrap.cc:654] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:36.794135 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 71de8356ea6f436fae0d65373e938232 (DEFAULT_TABLE table=test-table [id=bf32c9ddc0854d678222c3ae83f8b175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:36.794536 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71de8356ea6f436fae0d65373e938232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:36.794538 31289 log.cc:824] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:36.795369 31289 tablet_bootstrap.cc:492] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: No bootstrap required, opened a new log
I20260811 10:07:36.795444 31289 ts_tablet_manager.cc:1397] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:36.798053 31288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:36.798211 31288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:07:36.798458 31291 tablet_bootstrap.cc:492] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Bootstrap starting.
I20260811 10:07:36.798629 31182 tablet_service.cc:1467] Processing CreateTablet for tablet 71de8356ea6f436fae0d65373e938232 (DEFAULT_TABLE table=test-table [id=bf32c9ddc0854d678222c3ae83f8b175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:36.799028 31182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71de8356ea6f436fae0d65373e938232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:36.799146 31291 tablet_bootstrap.cc:654] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:36.799444 31291 log.cc:824] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:36.800328 31291 tablet_bootstrap.cc:492] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: No bootstrap required, opened a new log
I20260811 10:07:36.800416 31291 ts_tablet_manager.cc:1397] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:36.801829 31289 raft_consensus.cc:348] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.802066 31289 raft_consensus.cc:374] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:36.802136 31289 raft_consensus.cc:729] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2b0cc36c5b430b9d2f43181e10cbcf, State: Initialized, Role: FOLLOWER
I20260811 10:07:36.802356 31293 tablet_bootstrap.cc:492] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Bootstrap starting.
I20260811 10:07:36.802305 31289 consensus_queue.cc:260] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.802487 31291 raft_consensus.cc:348] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.802666 31291 raft_consensus.cc:374] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:36.802713 31291 raft_consensus.cc:729] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1262c1389b4425a6a42cb6df58fae6, State: Initialized, Role: FOLLOWER
I20260811 10:07:36.802771 31289 ts_tablet_manager.cc:1428] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260811 10:07:36.802865 31291 consensus_queue.cc:260] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.802918 30958 heartbeater.cc:503] Master 127.22.177.62:37093 was elected leader, sending a full tablet report...
I20260811 10:07:36.803123 31291 ts_tablet_manager.cc:1428] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:07:36.803248 31109 heartbeater.cc:503] Master 127.22.177.62:37093 was elected leader, sending a full tablet report...
I20260811 10:07:36.803949 31293 tablet_bootstrap.cc:654] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:36.804322 31293 log.cc:824] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:36.805248 31293 tablet_bootstrap.cc:492] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: No bootstrap required, opened a new log
I20260811 10:07:36.805328 31293 ts_tablet_manager.cc:1397] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:07:36.807317 31293 raft_consensus.cc:348] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.807480 31293 raft_consensus.cc:374] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:36.807503 31293 raft_consensus.cc:729] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de5bedd438b448b9992a005cdb498bb9, State: Initialized, Role: FOLLOWER
I20260811 10:07:36.807622 31293 consensus_queue.cc:260] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.807919 31293 ts_tablet_manager.cc:1428] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:07:36.807988 31253 heartbeater.cc:503] Master 127.22.177.62:37093 was elected leader, sending a full tablet report...
W20260811 10:07:36.848452 31111 tablet.cc:2369] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:36.938516 31296 raft_consensus.cc:482] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 10:07:36.938779 30960 tablet.cc:2369] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:36.938732 31296 raft_consensus.cc:504] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.939139 31296 leader_election.cc:288] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717), de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367)
I20260811 10:07:36.942368 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.939337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:40717 (local address 127.22.177.2:43681)
0811 10:07:36.939402 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.939416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.939503 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.939654 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.939655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.939658 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.939767 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.939774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.940671 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.940672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.941524 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.941527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.942110 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.942112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.942190 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.942193 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.942248 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.942279 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 10:07:36.942394 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.939460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:43681 (local address 127.22.177.1:40717)
0811 10:07:36.939513 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.939516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.939532 (+ 16us) server_negotiation.cc:407] Connection header received
0811 10:07:36.939567 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.939568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.939578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.939633 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.939851 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.940579 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.941669 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.942026 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.942085 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.942092 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.942273 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.942321 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 10:07:36.942497 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.939556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:54853 (local address 127.22.177.3:41367)
0811 10:07:36.939600 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.939602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.939611 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:36.939641 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.939642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.939652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.939705 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:36.939892 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.940530 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.941620 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.942084 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.942158 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.942167 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.942378 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.942424 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:07:36.942519 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.939485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41367 (local address 127.22.177.2:54853)
0811 10:07:36.939536 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:36.939546 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:36.939583 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:36.939727 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:36.939728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:36.939730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:36.939791 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.939792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.940612 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.940613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.941505 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:36.941507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.942181 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.942182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:36.942289 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.942292 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:36.942348 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:07:36.942386 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 10:07:36.942921 31202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de8356ea6f436fae0d65373e938232" candidate_uuid: "8b1262c1389b4425a6a42cb6df58fae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de5bedd438b448b9992a005cdb498bb9" is_pre_election: true
I20260811 10:07:36.942914 30905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de8356ea6f436fae0d65373e938232" candidate_uuid: "8b1262c1389b4425a6a42cb6df58fae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" is_pre_election: true
I20260811 10:07:36.943076 30905 raft_consensus.cc:2454] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1262c1389b4425a6a42cb6df58fae6 in term 0.
I20260811 10:07:36.943086 31202 raft_consensus.cc:2454] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1262c1389b4425a6a42cb6df58fae6 in term 0.
I20260811 10:07:36.943302 30982 leader_election.cc:302] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1262c1389b4425a6a42cb6df58fae6, ca2b0cc36c5b430b9d2f43181e10cbcf; no voters:
I20260811 10:07:36.943459 31296 raft_consensus.cc:2781] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:36.943536 31296 raft_consensus.cc:482] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:36.943574 31296 raft_consensus.cc:3037] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:36.944339 31296 raft_consensus.cc:504] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.944494 31296 leader_election.cc:288] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [CANDIDATE]: Term 1 election: Requested vote from peers ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717), de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367)
I20260811 10:07:36.944689 30905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de8356ea6f436fae0d65373e938232" candidate_uuid: "8b1262c1389b4425a6a42cb6df58fae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf"
I20260811 10:07:36.944716 31202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de8356ea6f436fae0d65373e938232" candidate_uuid: "8b1262c1389b4425a6a42cb6df58fae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de5bedd438b448b9992a005cdb498bb9"
I20260811 10:07:36.944792 31202 raft_consensus.cc:3037] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:36.944798 30905 raft_consensus.cc:3037] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:36.945644 30905 raft_consensus.cc:2454] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1262c1389b4425a6a42cb6df58fae6 in term 1.
I20260811 10:07:36.945570 31202 raft_consensus.cc:2454] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1262c1389b4425a6a42cb6df58fae6 in term 1.
I20260811 10:07:36.945855 30982 leader_election.cc:302] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1262c1389b4425a6a42cb6df58fae6, ca2b0cc36c5b430b9d2f43181e10cbcf; no voters:
I20260811 10:07:36.946017 31296 raft_consensus.cc:2781] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:36.946274 31296 raft_consensus.cc:686] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 LEADER]: Becoming Leader. State: Replica: 8b1262c1389b4425a6a42cb6df58fae6, State: Running, Role: LEADER
I20260811 10:07:36.946400 31296 consensus_queue.cc:237] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } }
I20260811 10:07:36.947319 29912 catalog_manager.cc:5495] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2). New cstate: current_term: 1 leader_uuid: "8b1262c1389b4425a6a42cb6df58fae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:36.971040 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.968408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.22.177.2:41947)
0811 10:07:36.968480 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.968482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.968491 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:36.968549 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.968550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.968577 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.968644 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:36.968805 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.969329 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.970154 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.970359 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.970424 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:36.970626 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:36.970742 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:36.970813 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:36.970916 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 10:07:36.970937 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:07:36.973130 31202 raft_consensus.cc:1261] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Refusing update from remote peer 8b1262c1389b4425a6a42cb6df58fae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:36.973178 30905 raft_consensus.cc:1261] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Refusing update from remote peer 8b1262c1389b4425a6a42cb6df58fae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:36.973532 31296 consensus_queue.cc:1035] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:36.973690 31299 consensus_queue.cc:1035] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:36.976873 31305 mvcc.cc:205] Tried to move back new op lower bound from 7317269942158708736 to 7317269942053036032. Current Snapshot: MvccSnapshot[applied={T|T < 7317269942158708736}]
I20260811 10:07:36.977733 31304 mvcc.cc:205] Tried to move back new op lower bound from 7317269942158708736 to 7317269942053036032. Current Snapshot: MvccSnapshot[applied={T|T < 7317269942158708736}]
I20260811 10:07:36.979185 29912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:36.979293 29912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:36.980743 29912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f5a9fbb091449f0bf7ff897bd7d6c8a: Sending DeleteTablet for 3 replicas of tablet 71de8356ea6f436fae0d65373e938232
I20260811 10:07:36.981132 30883 tablet_service.cc:1514] Processing DeleteTablet for tablet 71de8356ea6f436fae0d65373e938232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:36 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:53158
I20260811 10:07:36.981211 31030 tablet_service.cc:1514] Processing DeleteTablet for tablet 71de8356ea6f436fae0d65373e938232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:36 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:56126
I20260811 10:07:36.981352 31314 tablet_replica.cc:330] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: stopping tablet replica
I20260811 10:07:36.981459 31314 raft_consensus.cc:2229] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:36.981491 31315 tablet_replica.cc:330] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: stopping tablet replica
I20260811 10:07:36.981578 31314 raft_consensus.cc:2258] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:36.981552 31182 tablet_service.cc:1514] Processing DeleteTablet for tablet 71de8356ea6f436fae0d65373e938232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:36 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:43872
I20260811 10:07:36.981756 31315 raft_consensus.cc:2229] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:36.981873 31315 raft_consensus.cc:2258] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:36.982239 31316 tablet_replica.cc:330] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: stopping tablet replica
I20260811 10:07:36.982343 31316 raft_consensus.cc:2229] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:36.982441 31316 raft_consensus.cc:2258] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:36.982438 31315 ts_tablet_manager.cc:1905] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:36.982908 31316 ts_tablet_manager.cc:1905] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:36.983263 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 29876
I20260811 10:07:36.984615 31315 ts_tablet_manager.cc:1918] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:36.984716 31315 log.cc:1196] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/71de8356ea6f436fae0d65373e938232
I20260811 10:07:36.985090 31315 ts_tablet_manager.cc:1939] T 71de8356ea6f436fae0d65373e938232 P 8b1262c1389b4425a6a42cb6df58fae6: Deleting consensus metadata
I20260811 10:07:36.987668 31316 ts_tablet_manager.cc:1918] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:36.987775 31316 log.cc:1196] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/71de8356ea6f436fae0d65373e938232
I20260811 10:07:36.987890 31314 ts_tablet_manager.cc:1905] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:36.988065 31316 ts_tablet_manager.cc:1939] T 71de8356ea6f436fae0d65373e938232 P de5bedd438b448b9992a005cdb498bb9: Deleting consensus metadata
I20260811 10:07:36.989981 31314 ts_tablet_manager.cc:1918] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:36.990113 31314 log.cc:1196] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/71de8356ea6f436fae0d65373e938232
I20260811 10:07:36.990425 31314 ts_tablet_manager.cc:1939] T 71de8356ea6f436fae0d65373e938232 P ca2b0cc36c5b430b9d2f43181e10cbcf: Deleting consensus metadata
W20260811 10:07:36.994457 30984 connection.cc:535] server connection from 127.0.0.1:56126 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 10:07:36.994509 30832 connection.cc:535] server connection from 127.0.0.1:53158 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 10:07:36.994642 31134 connection.cc:535] server connection from 127.0.0.1:43872 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:07:37.005700 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.996702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.22.177.60:40081)
0811 10:07:36.996749 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.996751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.996887 (+ 136us) server_negotiation.cc:407] Connection header received
0811 10:07:36.996950 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.996951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.996962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.997015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.997586 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.998401 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.999462 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.999728 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.999808 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:37.001421 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:37.001423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:37.001423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:37.001439 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:37.004139 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:37.004679 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:37.004681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:37.004681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:37.004746 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:37.005004 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:37.005005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:37.005005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:37.005167 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:37.005284 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:37.005447 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 10:07:37.005546 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:07:37.005770 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:36.996646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.22.177.61:44937)
0811 10:07:36.996691 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:36.996693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:36.996749 (+ 56us) server_negotiation.cc:407] Connection header received
0811 10:07:36.996875 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:36.996876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:36.996895 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:36.996947 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:36.997736 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.998538 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.999256 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:36.999514 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:36.999579 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:37.001975 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:37.001977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:37.001977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:37.002004 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:37.004052 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:37.004744 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:37.004745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:37.004745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:37.004809 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:37.005058 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:37.005059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:37.005059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:37.005198 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:37.005295 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:37.005550 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 10:07:37.005666 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:07:37.978863 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:37.978548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.1:40257)
0811 10:07:37.978710 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:37.978820 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
I20260811 10:07:37.978852 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:37.978548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:47671)
0811 10:07:37.978692 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:37.978804 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
W20260811 10:07:37.979218 30958 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:37093 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
W20260811 10:07:37.979300 31253 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:37093 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:37.979436 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:37.979073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:37521)
0811 10:07:37.979182 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:37.979270 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
W20260811 10:07:37.979702 31109 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:37093 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:38.481205 31327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f5a9fbb091449f0bf7ff897bd7d6c8a)
I20260811 10:07:38.481366 31327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } }
I20260811 10:07:38.481766 31327 leader_election.cc:288] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093), 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60:40081)
I20260811 10:07:38.482039 31328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f5a9fbb091449f0bf7ff897bd7d6c8a)
I20260811 10:07:38.482179 31328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } }
I20260811 10:07:38.482254 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.481949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44692)
0811 10:07:38.482142 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.482202 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":58,"threads_started":1}
W20260811 10:07:38.482450 30540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:38.482841 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.482539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44698)
0811 10:07:38.482736 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.482798 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":56,"threads_started":1}
I20260811 10:07:38.482903 31328 leader_election.cc:288] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093), 369c82f45c084c0ba7e646b970b38f1e (127.22.177.61:44937)
W20260811 10:07:38.482957 30743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:38.483173 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.482832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44708)
0811 10:07:38.482875 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.483144 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 10:07:38.483321 30540 leader_election.cc:334] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093): Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:38.483551 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.483343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44714)
0811 10:07:38.483478 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.483521 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260811 10:07:38.483705 30743 leader_election.cc:334] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093): Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:38.485822 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.481966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:40081 (local address 127.0.0.1:35240)
0811 10:07:38.482188 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.482199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.482350 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.482613 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.482614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.482617 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.482689 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.482692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.483689 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.483689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.484826 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.484829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.485559 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.485560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.485672 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.485677 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.485733 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.485775 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":50,"threads_started":1}
I20260811 10:07:38.485864 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.482101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.22.177.60:40081)
0811 10:07:38.482286 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.482288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.482409 (+ 121us) server_negotiation.cc:407] Connection header received
0811 10:07:38.482441 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.482442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.482456 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.482545 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.482785 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.483595 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.484938 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.485470 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.485570 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.485582 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.485753 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.485799 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":121,"threads_started":1}
I20260811 10:07:38.486056 30770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "369c82f45c084c0ba7e646b970b38f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32f892a7e1e442669aeff9c58f79e3d9" is_pre_election: true
I20260811 10:07:38.486181 30770 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 369c82f45c084c0ba7e646b970b38f1e in term 1.
I20260811 10:07:38.486418 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.482603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:44937 (local address 127.0.0.1:34870)
0811 10:07:38.483163 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.483173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.483279 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.483491 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.483492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.483495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.483536 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.483538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.484333 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.484334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.485370 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.485373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.486156 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.486157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.486234 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.486238 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.486326 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.486373 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":85,"threads_started":1}
I20260811 10:07:38.486426 30541 leader_election.cc:302] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32f892a7e1e442669aeff9c58f79e3d9, 369c82f45c084c0ba7e646b970b38f1e; no voters: 3f5a9fbb091449f0bf7ff897bd7d6c8a
I20260811 10:07:38.486622 31327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 10:07:38.486752 31327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f5a9fbb091449f0bf7ff897bd7d6c8a)
I20260811 10:07:38.486788 31327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:07:38.486833 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.482778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.22.177.61:44937)
0811 10:07:38.483027 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.483029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.483247 (+ 218us) server_negotiation.cc:407] Connection header received
0811 10:07:38.483359 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.483360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.483370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.483418 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.483622 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.484244 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.485453 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.486062 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.486144 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.486155 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.486571 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.486620 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
I20260811 10:07:38.487118 30568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32f892a7e1e442669aeff9c58f79e3d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "369c82f45c084c0ba7e646b970b38f1e" is_pre_election: true
I20260811 10:07:38.487684 31327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } }
I20260811 10:07:38.487824 30568 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32f892a7e1e442669aeff9c58f79e3d9 in current term 2: Already voted for candidate 369c82f45c084c0ba7e646b970b38f1e in this term.
I20260811 10:07:38.487854 31327 leader_election.cc:288] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [CANDIDATE]: Term 2 election: Requested vote from peers 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093), 32f892a7e1e442669aeff9c58f79e3d9 (127.22.177.60:40081)
I20260811 10:07:38.488062 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.487926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44730)
0811 10:07:38.488004 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.488033 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 10:07:38.488063 30742 leader_election.cc:302] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32f892a7e1e442669aeff9c58f79e3d9; no voters: 369c82f45c084c0ba7e646b970b38f1e, 3f5a9fbb091449f0bf7ff897bd7d6c8a
I20260811 10:07:38.488046 30770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "369c82f45c084c0ba7e646b970b38f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32f892a7e1e442669aeff9c58f79e3d9"
I20260811 10:07:38.488164 30770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:07:38.489029 30770 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 369c82f45c084c0ba7e646b970b38f1e in term 2.
I20260811 10:07:38.489176 31328 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:07:38.489202 30541 leader_election.cc:302] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32f892a7e1e442669aeff9c58f79e3d9, 369c82f45c084c0ba7e646b970b38f1e; no voters:
I20260811 10:07:38.489287 31327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:07:38.489436 31327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [term 2 LEADER]: Becoming Leader. State: Replica: 369c82f45c084c0ba7e646b970b38f1e, State: Running, Role: LEADER
I20260811 10:07:38.489445 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.489364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44754)
0811 10:07:38.489397 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.489421 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 10:07:38.489531 31327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } }
W20260811 10:07:38.489562 30540 leader_election.cc:334] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093): Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
I20260811 10:07:38.489946 31337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369c82f45c084c0ba7e646b970b38f1e. Latest consensus state: current_term: 2 leader_uuid: "369c82f45c084c0ba7e646b970b38f1e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:38.490018 31337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:38.490182 31340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:38.490530 31340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:38.490583 31340 catalog_manager.cc:1259] Loaded cluster ID: 1dc09f59a80048908d229b2985f3ca62
I20260811 10:07:38.490600 31340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:38.490722 31340 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:38.490777 31340 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Loaded TSK: 0
I20260811 10:07:38.490873 31340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:07:38.546929 30557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:38.548700 30557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:38.549973 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.549842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44774)
0811 10:07:38.549908 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.549944 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 10:07:38.549978 30770 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [term 2 FOLLOWER]: Refusing update from remote peer 369c82f45c084c0ba7e646b970b38f1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 10:07:38.550441 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.550278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.0.0.1:44776)
0811 10:07:38.550333 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.550359 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 10:07:38.550369 31337 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 10:07:38.550559 30540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e -> Peer 3f5a9fbb091449f0bf7ff897bd7d6c8a (127.22.177.62:37093): Couldn't send request to peer 3f5a9fbb091449f0bf7ff897bd7d6c8a. Status: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 10:07:38.551100 31328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369c82f45c084c0ba7e646b970b38f1e. Latest consensus state: current_term: 2 leader_uuid: "369c82f45c084c0ba7e646b970b38f1e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:38.551239 31328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:07:38.551755 31327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "369c82f45c084c0ba7e646b970b38f1e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:38.551846 31327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:38.552462 31328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "369c82f45c084c0ba7e646b970b38f1e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3f5a9fbb091449f0bf7ff897bd7d6c8a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37093 } } peers { permanent_uuid: "369c82f45c084c0ba7e646b970b38f1e" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 44937 } attrs { promote: false } } peers { permanent_uuid: "32f892a7e1e442669aeff9c58f79e3d9" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 40081 } attrs { promote: false } } }
I20260811 10:07:38.552521 31342 mvcc.cc:205] Tried to move back new op lower bound from 7317269948618170368 to 7317269948373876736. Current Snapshot: MvccSnapshot[applied={T|T < 7317269948618170368}]
I20260811 10:07:38.552579 31328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f892a7e1e442669aeff9c58f79e3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:07:38.557269 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.554097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.22.177.1:40717)
0811 10:07:38.554328 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.554331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.554343 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:38.554375 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.554376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.554389 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.554489 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.554627 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.555192 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.556309 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.556883 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.556945 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.556960 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.557133 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.557187 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260811 10:07:38.557279 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.554004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:40717 (local address 127.0.0.1:49236)
0811 10:07:38.554038 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.554051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.554107 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.554495 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.554496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.554497 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.554547 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.554549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.555292 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.555293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.556217 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.556219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.556990 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.556991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.557048 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.557053 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.557107 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.557202 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 10:07:38.557847 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.554103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:41367 (local address 127.0.0.1:60216)
0811 10:07:38.554321 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.554334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.554397 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.554601 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.554602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.554604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.554658 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.554659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.555609 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.555610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.556771 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.556774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.557569 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.557570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.557661 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.557666 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.557715 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.557771 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":28}
I20260811 10:07:38.557873 30883 tablet_service.cc:1467] Processing CreateTablet for tablet b9bfeef52b884cebac2c8b45c1f096c4 (DEFAULT_TABLE table=test-table [id=9679a6745290426a9600f7e1ff2189ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:38.557925 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.554308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.22.177.2:41947)
0811 10:07:38.554740 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.554742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.554752 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:38.554778 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.554779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.554790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.554988 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.555042 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.555737 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.556942 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.557595 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.557700 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.557715 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.557802 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.557847 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":56,"threads_started":1}
I20260811 10:07:38.557937 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.554114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:41947 (local address 127.0.0.1:40402)
0811 10:07:38.554566 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.554576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.554620 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.554889 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.554890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.554893 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.554926 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.554927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.555825 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.555826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.556871 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.556874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.557687 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.557688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.557744 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.557747 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.557798 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.557834 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":93,"threads_started":1}
I20260811 10:07:38.558033 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9bfeef52b884cebac2c8b45c1f096c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:38.558094 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.554322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.22.177.3:41367)
0811 10:07:38.554424 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.554426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.554438 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:38.554466 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.554467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.554479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.554545 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.554756 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.555518 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.556866 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.557470 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.557557 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.557570 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.557953 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.558011 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":57,"threads_started":1}
I20260811 10:07:38.559232 31351 tablet_bootstrap.cc:492] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf: Bootstrap starting.
I20260811 10:07:38.559234 31030 tablet_service.cc:1467] Processing CreateTablet for tablet b9bfeef52b884cebac2c8b45c1f096c4 (DEFAULT_TABLE table=test-table [id=9679a6745290426a9600f7e1ff2189ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:38.559338 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9bfeef52b884cebac2c8b45c1f096c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:38.559578 31351 tablet_bootstrap.cc:654] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:38.560190 31351 tablet_bootstrap.cc:492] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf: No bootstrap required, opened a new log
I20260811 10:07:38.560308 31351 ts_tablet_manager.cc:1397] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:38.560439 31351 raft_consensus.cc:348] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.560494 31351 raft_consensus.cc:374] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:38.560508 31351 raft_consensus.cc:729] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2b0cc36c5b430b9d2f43181e10cbcf, State: Initialized, Role: FOLLOWER
I20260811 10:07:38.560513 31353 tablet_bootstrap.cc:492] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Bootstrap starting.
I20260811 10:07:38.560612 31351 consensus_queue.cc:260] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.560781 31351 ts_tablet_manager.cc:1428] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:38.560889 31182 tablet_service.cc:1467] Processing CreateTablet for tablet b9bfeef52b884cebac2c8b45c1f096c4 (DEFAULT_TABLE table=test-table [id=9679a6745290426a9600f7e1ff2189ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:38.560945 31353 tablet_bootstrap.cc:654] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:38.561028 31182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9bfeef52b884cebac2c8b45c1f096c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:38.561621 31353 tablet_bootstrap.cc:492] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: No bootstrap required, opened a new log
I20260811 10:07:38.561669 31353 ts_tablet_manager.cc:1397] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:38.561792 31353 raft_consensus.cc:348] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.561867 31353 raft_consensus.cc:374] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:38.561893 31353 raft_consensus.cc:729] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1262c1389b4425a6a42cb6df58fae6, State: Initialized, Role: FOLLOWER
I20260811 10:07:38.561966 31353 consensus_queue.cc:260] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.562114 31353 ts_tablet_manager.cc:1428] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:38.562523 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.562303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:59661)
0811 10:07:38.562443 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.562486 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":91,"threads_started":1}
I20260811 10:07:38.562523 31359 tablet_bootstrap.cc:492] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Bootstrap starting.
I20260811 10:07:38.563189 31359 tablet_bootstrap.cc:654] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:38.563515 31110 heartbeater.cc:503] Master 127.22.177.61:44937 was elected leader, sending a full tablet report...
I20260811 10:07:38.563578 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.562767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:45483)
0811 10:07:38.563468 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.563501 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":682}
I20260811 10:07:38.563848 31359 tablet_bootstrap.cc:492] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: No bootstrap required, opened a new log
I20260811 10:07:38.563911 31359 ts_tablet_manager.cc:1397] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:38.564059 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.561019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.1:58297)
0811 10:07:38.563879 (+ 2860us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.564015 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2837,"thread_start_us":71,"threads_started":1}
I20260811 10:07:38.564059 31359 raft_consensus.cc:348] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.564127 31359 raft_consensus.cc:374] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:38.564152 31359 raft_consensus.cc:729] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de5bedd438b448b9992a005cdb498bb9, State: Initialized, Role: FOLLOWER
I20260811 10:07:38.564230 31359 consensus_queue.cc:260] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.564416 31359 ts_tablet_manager.cc:1428] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:38.564845 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.564621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:53831)
0811 10:07:38.564771 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.564809 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":112,"threads_started":1}
I20260811 10:07:38.565129 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.565054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:46851)
0811 10:07:38.565085 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.565108 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260811 10:07:38.748464 31361 raft_consensus.cc:482] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:38.748592 31361 raft_consensus.cc:504] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.749097 31361 leader_election.cc:288] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717), 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947)
I20260811 10:07:38.752562 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.749387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:35153 (local address 127.22.177.1:40717)
0811 10:07:38.749440 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.749442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.749676 (+ 234us) server_negotiation.cc:407] Connection header received
0811 10:07:38.749741 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.749742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.749755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.749828 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.750042 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.750584 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.751753 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.752138 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.752291 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.752304 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.752457 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.752500 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:07:38.752563 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.749286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:40717 (local address 127.22.177.3:35153)
0811 10:07:38.749567 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.749578 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.749681 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.749863 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.749864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.749866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.749932 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.749934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.750678 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.750679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.751613 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.751616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.752292 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.752292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.752372 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.752376 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.752446 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.752487 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":122,"thread_start_us":149,"threads_started":1}
I20260811 10:07:38.752636 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.749418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:56119 (local address 127.22.177.2:41947)
0811 10:07:38.749498 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.749500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.749510 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:38.749552 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.749554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.749564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.749669 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:38.749882 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.750593 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.751862 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.752234 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.752342 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.752360 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.752524 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.752565 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:07:38.752874 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.749281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:41947 (local address 127.22.177.3:56119)
0811 10:07:38.749375 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.749390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:38.749470 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:38.749692 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:38.749693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:38.749697 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:38.749790 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.749793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.750699 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.750700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.751765 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:38.751768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.752346 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.752347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:38.752430 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.752433 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:38.752511 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 10:07:38.752553 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":28}
I20260811 10:07:38.752876 30905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bfeef52b884cebac2c8b45c1f096c4" candidate_uuid: "de5bedd438b448b9992a005cdb498bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" is_pre_election: true
I20260811 10:07:38.752964 30905 raft_consensus.cc:2454] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de5bedd438b448b9992a005cdb498bb9 in term 0.
W20260811 10:07:38.753029 31255 tablet.cc:2369] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:38.753147 31133 leader_election.cc:302] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2b0cc36c5b430b9d2f43181e10cbcf, de5bedd438b448b9992a005cdb498bb9; no voters:
I20260811 10:07:38.753142 31050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bfeef52b884cebac2c8b45c1f096c4" candidate_uuid: "de5bedd438b448b9992a005cdb498bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1262c1389b4425a6a42cb6df58fae6" is_pre_election: true
I20260811 10:07:38.753273 31050 raft_consensus.cc:2454] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de5bedd438b448b9992a005cdb498bb9 in term 0.
I20260811 10:07:38.753284 31361 raft_consensus.cc:2781] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:38.753340 31361 raft_consensus.cc:482] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:38.753362 31361 raft_consensus.cc:3037] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:38.754123 31361 raft_consensus.cc:504] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.754276 31361 leader_election.cc:288] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [CANDIDATE]: Term 1 election: Requested vote from peers ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717), 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947)
I20260811 10:07:38.754438 30905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bfeef52b884cebac2c8b45c1f096c4" candidate_uuid: "de5bedd438b448b9992a005cdb498bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf"
I20260811 10:07:38.754479 31050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bfeef52b884cebac2c8b45c1f096c4" candidate_uuid: "de5bedd438b448b9992a005cdb498bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1262c1389b4425a6a42cb6df58fae6"
I20260811 10:07:38.754544 30905 raft_consensus.cc:3037] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:38.754549 31050 raft_consensus.cc:3037] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:38.755198 30905 raft_consensus.cc:2454] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de5bedd438b448b9992a005cdb498bb9 in term 1.
I20260811 10:07:38.755414 31133 leader_election.cc:302] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2b0cc36c5b430b9d2f43181e10cbcf, de5bedd438b448b9992a005cdb498bb9; no voters:
I20260811 10:07:38.755417 31050 raft_consensus.cc:2454] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de5bedd438b448b9992a005cdb498bb9 in term 1.
I20260811 10:07:38.755509 31361 raft_consensus.cc:2781] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:38.755664 31361 raft_consensus.cc:686] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 1 LEADER]: Becoming Leader. State: Replica: de5bedd438b448b9992a005cdb498bb9, State: Running, Role: LEADER
I20260811 10:07:38.755756 31361 consensus_queue.cc:237] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } }
I20260811 10:07:38.756130 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.756019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:59053)
0811 10:07:38.756071 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.756107 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 10:07:38.756238 31254 heartbeater.cc:503] Master 127.22.177.61:44937 was elected leader, sending a full tablet report...
I20260811 10:07:38.756649 30556 catalog_manager.cc:5495] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to de5bedd438b448b9992a005cdb498bb9 (127.22.177.3). New cstate: current_term: 1 leader_uuid: "de5bedd438b448b9992a005cdb498bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de5bedd438b448b9992a005cdb498bb9" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:38.787034 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.784378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.22.177.3:41367)
0811 10:07:38.784449 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:38.784451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:38.784460 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:38.784497 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:38.784498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:38.784518 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:38.784582 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:38.784838 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.785313 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.786206 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:38.786411 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:38.786484 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:38.786666 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:38.786772 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:38.786835 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:38.786882 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 10:07:38.786907 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 10:07:38.788528 30905 raft_consensus.cc:1261] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Refusing update from remote peer de5bedd438b448b9992a005cdb498bb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:38.788544 31050 raft_consensus.cc:1261] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Refusing update from remote peer de5bedd438b448b9992a005cdb498bb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:38.788875 31361 consensus_queue.cc:1035] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2b0cc36c5b430b9d2f43181e10cbcf" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 40717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:38.789028 31364 consensus_queue.cc:1035] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b1262c1389b4425a6a42cb6df58fae6" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:38.789479 30959 heartbeater.cc:503] Master 127.22.177.61:44937 was elected leader, sending a full tablet report...
I20260811 10:07:38.790117 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.789966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.1:42031)
0811 10:07:38.790038 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.790077 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 10:07:38.790407 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.790292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:33411)
0811 10:07:38.790339 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.790373 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 10:07:38.791772 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.791540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:52581)
0811 10:07:38.791709 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.791740 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 10:07:38.792038 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.791661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.1:46323)
0811 10:07:38.791979 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.792010 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301}
I20260811 10:07:38.792565 31369 mvcc.cc:205] Tried to move back new op lower bound from 7317269949595131904 to 7317269949464043520. Current Snapshot: MvccSnapshot[applied={T|T < 7317269949595131904 or (T in {7317269949595131904})}]
I20260811 10:07:38.792940 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.792591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.2:47103)
0811 10:07:38.792631 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.792903 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 10:07:38.793660 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:38.793526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37093 (local address 127.22.177.3:55625)
0811 10:07:38.793560 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.793621 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:37093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 10:07:38.794237 30557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:38.794325 30557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:38.795742 30557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 369c82f45c084c0ba7e646b970b38f1e: Sending DeleteTablet for 3 replicas of tablet b9bfeef52b884cebac2c8b45c1f096c4
I20260811 10:07:38.796111 30883 tablet_service.cc:1514] Processing DeleteTablet for tablet b9bfeef52b884cebac2c8b45c1f096c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:38 UTC) from {username='oryx', principal='oryx/127.22.177.61@KRBTEST.COM'} at 127.0.0.1:49236
I20260811 10:07:38.796164 31030 tablet_service.cc:1514] Processing DeleteTablet for tablet b9bfeef52b884cebac2c8b45c1f096c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:38 UTC) from {username='oryx', principal='oryx/127.22.177.61@KRBTEST.COM'} at 127.0.0.1:40402
I20260811 10:07:38.796376 31182 tablet_service.cc:1514] Processing DeleteTablet for tablet b9bfeef52b884cebac2c8b45c1f096c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:38 UTC) from {username='oryx', principal='oryx/127.22.177.61@KRBTEST.COM'} at 127.0.0.1:60216
I20260811 10:07:38.796469 31379 tablet_replica.cc:330] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf: stopping tablet replica
I20260811 10:07:38.796523 31379 raft_consensus.cc:2229] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:38.796578 31379 raft_consensus.cc:2258] T b9bfeef52b884cebac2c8b45c1f096c4 P ca2b0cc36c5b430b9d2f43181e10cbcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:38.796685 31380 tablet_replica.cc:330] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: stopping tablet replica
I20260811 10:07:38.796692 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 30816
I20260811 10:07:38.796749 31380 raft_consensus.cc:2229] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:38.796802 31380 raft_consensus.cc:2258] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:38.798159 31380 ts_tablet_manager.cc:1905] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:38.800556 31380 ts_tablet_manager.cc:1918] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:38.800627 31380 log.cc:1196] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/b9bfeef52b884cebac2c8b45c1f096c4
I20260811 10:07:38.800899 31380 ts_tablet_manager.cc:1939] T b9bfeef52b884cebac2c8b45c1f096c4 P 8b1262c1389b4425a6a42cb6df58fae6: Deleting consensus metadata
I20260811 10:07:38.801280 30542 catalog_manager.cc:4841] TS 8b1262c1389b4425a6a42cb6df58fae6 (127.22.177.2:41947): tablet b9bfeef52b884cebac2c8b45c1f096c4 (table test-table [id=9679a6745290426a9600f7e1ff2189ac]) successfully deleted
I20260811 10:07:38.805938 31381 tablet_replica.cc:330] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: stopping tablet replica
I20260811 10:07:38.806048 31381 raft_consensus.cc:2229] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:38.806226 31381 raft_consensus.cc:2258] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:38.806604 31381 ts_tablet_manager.cc:1905] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:38.808213 31381 ts_tablet_manager.cc:1918] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 10:07:38.808233 31330 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:38.808072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:40717 (local address 127.0.0.1:49238)
0811 10:07:38.808131 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:38.808163 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:40717: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:07:38.808279 31381 log.cc:1196] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/b9bfeef52b884cebac2c8b45c1f096c4
W20260811 10:07:38.808408 30540 catalog_manager.cc:4584] TS ca2b0cc36c5b430b9d2f43181e10cbcf (127.22.177.1:40717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9bfeef52b884cebac2c8b45c1f096c4: Network error: Client connection negotiation failed: client connection to 127.22.177.1:40717: connect: Connection reset by peer (error 104)
I20260811 10:07:38.808460 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 30964
I20260811 10:07:38.812431 31381 ts_tablet_manager.cc:1939] T b9bfeef52b884cebac2c8b45c1f096c4 P de5bedd438b448b9992a005cdb498bb9: Deleting consensus metadata
I20260811 10:07:38.812886 30540 catalog_manager.cc:4841] TS de5bedd438b448b9992a005cdb498bb9 (127.22.177.3:41367): tablet b9bfeef52b884cebac2c8b45c1f096c4 (table test-table [id=9679a6745290426a9600f7e1ff2189ac]) successfully deleted
I20260811 10:07:38.818886 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 31116
I20260811 10:07:38.828048 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 30411
I20260811 10:07:38.836663 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 30614
2026-08-11T10:07:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5982 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:38 dist-test-slave-g350 krb5kdc[31385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:38 dist-test-slave-g350 krb5kdc[31385](info): set up 2 sockets
Aug 11 10:07:38 dist-test-slave-g350 krb5kdc[31385](info): commencing operation
krb5kdc: starting...
W20260811 10:07:39.971549 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab.
Entry for principal oryx/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab.
I20260811 10:07:40.109566 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35837
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab
--principal=oryx/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:35837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:40.194527 31399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:40.194764 31399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:40.194803 31399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:40.196028 31399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:40.196094 31399 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 10:07:40.196118 31399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:40.196148 31399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:40.196179 31399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:40.196216 31399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:40.197753 31399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:35837
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.62.keytab
--principal=oryx/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:40.198021 31399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:40.198256 31399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:40.203271 31406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:40.203285 31408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:40.203403 31399 server_base.cc:1034] running on GCE node
W20260811 10:07:40.203512 31405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:40.203801 31399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:40.204089 31399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:40.205284 31399 hybrid_clock.cc:648] HybridClock initialized: now 1786442860205255 us; error 43 us; skew 500 ppm
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:40.214413 31399 init.cc:377] Logged in from keytab as oryx/127.22.177.62@KRBTEST.COM (short username oryx)
I20260811 10:07:40.215005 31399 webserver.cc:460] Webserver started at http://127.22.177.62:38727/ using document root <none> and password file <none>
I20260811 10:07:40.215262 31399 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:40.215312 31399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:40.215466 31399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:40.216614 31399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "d10448ece2b8459b9a369e5714a6684b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "b515bf33068c179cd8fb4c8f2ab868aa"
server_key_iv: "b64c5658b0ff5c86f802fb1e986349ca"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.217033 31399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "d10448ece2b8459b9a369e5714a6684b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "b515bf33068c179cd8fb4c8f2ab868aa"
server_key_iv: "b64c5658b0ff5c86f802fb1e986349ca"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.218781 31399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:07:40.219671 31416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.220042 31399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:40.220175 31399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "d10448ece2b8459b9a369e5714a6684b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "b515bf33068c179cd8fb4c8f2ab868aa"
server_key_iv: "b64c5658b0ff5c86f802fb1e986349ca"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.220268 31399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:40.231642 31399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:40.234234 31399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:40.234403 31399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:40.239763 31399 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:35837
I20260811 10:07:40.239809 31484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:35837 every 8 connection(s)
I20260811 10:07:40.240157 31399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:40.240933 31485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.243515 31485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Bootstrap starting.
I20260811 10:07:40.244156 31485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.244477 31485 log.cc:824] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:40.245191 31485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: No bootstrap required, opened a new log
I20260811 10:07:40.245812 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 31399
I20260811 10:07:40.245908 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:40.246145 23236 external_mini_cluster.cc:1428] Setting key 9f3f95192ca63db6f2d166a500924280
I20260811 10:07:40.247682 31485 raft_consensus.cc:348] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10448ece2b8459b9a369e5714a6684b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35837 } }
I20260811 10:07:40.247874 31485 raft_consensus.cc:374] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.247915 31485 raft_consensus.cc:729] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10448ece2b8459b9a369e5714a6684b, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.248046 31485 consensus_queue.cc:260] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10448ece2b8459b9a369e5714a6684b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35837 } }
I20260811 10:07:40.248132 31485 raft_consensus.cc:388] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:40.248157 31485 raft_consensus.cc:482] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:40.248224 31485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:40.248915 31485 raft_consensus.cc:504] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10448ece2b8459b9a369e5714a6684b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35837 } }
I20260811 10:07:40.249030 31485 leader_election.cc:302] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d10448ece2b8459b9a369e5714a6684b; no voters:
I20260811 10:07:40.249192 31485 leader_election.cc:288] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:40.249277 31491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:40.249451 31485 sys_catalog.cc:564] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:40.249482 31491 raft_consensus.cc:686] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [term 1 LEADER]: Becoming Leader. State: Replica: d10448ece2b8459b9a369e5714a6684b, State: Running, Role: LEADER
I20260811 10:07:40.249577 31491 consensus_queue.cc:237] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10448ece2b8459b9a369e5714a6684b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35837 } }
I20260811 10:07:40.249963 31492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d10448ece2b8459b9a369e5714a6684b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10448ece2b8459b9a369e5714a6684b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35837 } } }
I20260811 10:07:40.249945 31491 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d10448ece2b8459b9a369e5714a6684b. Latest consensus state: current_term: 1 leader_uuid: "d10448ece2b8459b9a369e5714a6684b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10448ece2b8459b9a369e5714a6684b" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35837 } } }
I20260811 10:07:40.250190 31491 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:40.250473 31492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:40.250785 31500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:40.251600 31500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:40.253305 31500 catalog_manager.cc:1347] Generated new cluster ID: 818c0b70606c4c55b0065f2bc212be0e
I20260811 10:07:40.253366 31500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:40.259974 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.246829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.22.177.62:35837)
0811 10:07:40.247362 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.247372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.247406 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:07:40.247639 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.247648 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.247712 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.247865 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.248685 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.250158 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.251303 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.251895 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.252337 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.255041 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.255046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.255046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.255058 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.257975 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.258400 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.258403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.258403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.258510 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.258696 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.258697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.258697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.258987 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.259543 (+ 556us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.259613 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.259746 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260811 10:07:40.265838 31500 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:40.266557 31500 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:40.269909 31500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Generated new TSK 0
I20260811 10:07:40.270112 31500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab.
Entry for principal oryx/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab.
I20260811 10:07:40.303274 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:35837
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:40.388746 31512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:40.388944 31512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:40.388968 31512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:40.390177 31512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:40.390218 31512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:40.390239 31512 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:40.390288 31512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:40.392004 31512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:35837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:40.392267 31512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:40.392513 31512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:40.397931 31521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:40.397927 31519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:40.398077 31512 server_base.cc:1034] running on GCE node
W20260811 10:07:40.397969 31518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:40.398478 31512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:40.398734 31512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:40.399902 31512 hybrid_clock.cc:648] HybridClock initialized: now 1786442860399887 us; error 33 us; skew 500 ppm
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:40.402643 31512 init.cc:377] Logged in from keytab as oryx/127.22.177.1@KRBTEST.COM (short username oryx)
I20260811 10:07:40.403115 31512 webserver.cc:460] Webserver started at http://127.22.177.1:38153/ using document root <none> and password file <none>
I20260811 10:07:40.403326 31512 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:40.403371 31512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:40.403461 31512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:40.404452 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "17cf266dba5d4e4abd29e32b8f501ff4"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "d4984606445be35b063130b490f823c4"
server_key_iv: "624e9db21abd3cf0ffc5dd38f79cd8d0"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.404740 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "17cf266dba5d4e4abd29e32b8f501ff4"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "d4984606445be35b063130b490f823c4"
server_key_iv: "624e9db21abd3cf0ffc5dd38f79cd8d0"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.406294 31512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:40.407173 31528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.407409 31512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:40.407510 31512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "17cf266dba5d4e4abd29e32b8f501ff4"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "d4984606445be35b063130b490f823c4"
server_key_iv: "624e9db21abd3cf0ffc5dd38f79cd8d0"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.407577 31512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:40.415273 31512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:40.417553 31512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:40.417850 31512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:40.418357 31512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:40.418406 31512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.418432 31512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:40.418483 31512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.426556 31512 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:46523
I20260811 10:07:40.426646 31656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:46523 every 8 connection(s)
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:40.427031 31512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:40.429225 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 31512
I20260811 10:07:40.429355 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:40.429651 23236 external_mini_cluster.cc:1428] Setting key feb26c2c6e71c9712c1b1a9ebad209ee
I20260811 10:07:40.430625 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.418932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:53863 (local address 127.22.177.62:35837)
0811 10:07:40.418999 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.419002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.419312 (+ 310us) server_negotiation.cc:407] Connection header received
0811 10:07:40.419871 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.419872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.419889 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.419968 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.420957 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.421837 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.423698 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.424056 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.424133 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.427346 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.427347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.427347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.427359 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.428937 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.429497 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.429497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.429498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.429552 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.429887 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.429888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.429889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.430044 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.430114 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.430477 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.430563 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:07:40.430943 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.418897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.22.177.1:53863)
0811 10:07:40.419142 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.419171 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.419754 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.420094 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.420099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.420514 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:40.420832 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.420839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.421948 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.421949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.423557 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.423561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.424213 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.424214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.424378 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.425145 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:40.425155 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:40.427203 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:40.429058 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.429067 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.429069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.429418 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.429657 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.429658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.429659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.429805 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.430118 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:40.430124 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:40.430311 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.430449 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.430664 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":81,"threads_started":1}
I20260811 10:07:40.431638 31657 heartbeater.cc:347] Connected to a master server at 127.22.177.62:35837
I20260811 10:07:40.431735 31657 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:40.431947 31657 heartbeater.cc:511] Master 127.22.177.62:35837 requested a full tablet report, sending...
I20260811 10:07:40.432545 31432 ts_manager.cc:194] Registered new tserver with Master: 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523)
I20260811 10:07:40.433398 31432 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:53863
I20260811 10:07:40.439339 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.432941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:52447 (local address 127.22.177.62:35837)
0811 10:07:40.432989 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.432991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.433155 (+ 164us) server_negotiation.cc:407] Connection header received
0811 10:07:40.433218 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.433219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.433229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.433284 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.433776 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.434480 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.435185 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.435514 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.435599 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.436606 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.436607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.436607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.436613 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.438146 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.438561 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.438562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.438562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.438620 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.438849 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.438850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.438850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.438959 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.439013 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.439196 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.439266 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:07:40.439424 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.432869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.22.177.1:52447)
0811 10:07:40.433059 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.433072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.433151 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.433302 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.433302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.433544 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:40.433662 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.433664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.434592 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.434593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.435090 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.435091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.435614 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.435615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.435674 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.436195 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:40.436200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:40.436517 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:40.438233 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.438234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.438235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.438461 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.438701 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.438702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.438702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.438781 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.439029 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:40.439030 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:40.439084 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.439181 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.439297 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab.
Entry for principal oryx/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab.
I20260811 10:07:40.461826 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:35837
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:40.547636 31666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:40.547870 31666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:40.547897 31666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:40.549201 31666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:40.549248 31666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:40.549271 31666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:40.549289 31666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:40.551155 31666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:35837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:40.551466 31666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:40.551729 31666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:07:40.557540 31666 server_base.cc:1034] running on GCE node
W20260811 10:07:40.557472 31671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:40.557509 31672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:40.557502 31674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:40.558059 31666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:40.558382 31666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:40.559566 31666 hybrid_clock.cc:648] HybridClock initialized: now 1786442860559565 us; error 45 us; skew 500 ppm
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:40.562417 31666 init.cc:377] Logged in from keytab as oryx/127.22.177.2@KRBTEST.COM (short username oryx)
I20260811 10:07:40.562942 31666 webserver.cc:460] Webserver started at http://127.22.177.2:34615/ using document root <none> and password file <none>
I20260811 10:07:40.563207 31666 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:40.563279 31666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:40.563431 31666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:40.564536 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "13b31f041c84c78661fab0d894fd4830"
server_key_iv: "7c86bd08a5d199807aed8ee1ee11cf5a"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.564909 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "13b31f041c84c78661fab0d894fd4830"
server_key_iv: "7c86bd08a5d199807aed8ee1ee11cf5a"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.566417 31666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:07:40.567318 31681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.567571 31666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:40.567726 31666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "13b31f041c84c78661fab0d894fd4830"
server_key_iv: "7c86bd08a5d199807aed8ee1ee11cf5a"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.567814 31666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:40.584233 31666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:40.586722 31666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:40.586920 31666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:40.587585 31666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:40.587638 31666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.587668 31666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:40.587700 31666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:40.595180 31666 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:39677
I20260811 10:07:40.595564 31666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:40.595669 31807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:39677 every 8 connection(s)
I20260811 10:07:40.598279 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 31666
I20260811 10:07:40.598382 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.588025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:58737 (local address 127.22.177.62:35837)
0811 10:07:40.588080 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.588083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.588333 (+ 250us) server_negotiation.cc:407] Connection header received
0811 10:07:40.588867 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.588869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.588884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.588939 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.589845 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.590356 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.591413 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.591610 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.591689 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.595230 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.595232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.595232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.595242 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.596847 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.597299 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.597300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.597301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.597362 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.597648 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.597649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.597649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.597787 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.597869 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.598212 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.598286 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:07:40.598394 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:40.598604 31690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.587974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.22.177.2:58737)
0811 10:07:40.588128 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.588179 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.588750 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.589011 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.589017 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.589397 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:40.589716 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.589722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.590462 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.590464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.591307 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.591310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.591700 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.591701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.591845 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.592453 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:40.592464 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:40.595121 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:40.596958 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.596967 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.596969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.597206 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.597443 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.597443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.597444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.597553 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.597861 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:40.597869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:40.598012 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.598144 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.598346 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260811 10:07:40.598629 23236 external_mini_cluster.cc:1428] Setting key 3999352e36aeedac4bd09af2bed7621a
I20260811 10:07:40.599120 31808 heartbeater.cc:347] Connected to a master server at 127.22.177.62:35837
I20260811 10:07:40.599200 31808 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:40.599380 31808 heartbeater.cc:511] Master 127.22.177.62:35837 requested a full tablet report, sending...
I20260811 10:07:40.599743 31431 ts_manager.cc:194] Registered new tserver with Master: 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677)
I20260811 10:07:40.600255 31431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:58737
I20260811 10:07:40.607331 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.600251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:46787 (local address 127.22.177.62:35837)
0811 10:07:40.600292 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.600294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.601510 (+ 1216us) server_negotiation.cc:407] Connection header received
0811 10:07:40.601573 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.601574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.601585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.601687 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.602015 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.602483 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.603199 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.603441 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.603514 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.604473 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.604475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.604475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.604482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.606058 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.606441 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.606443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.606443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.606501 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.606730 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.606731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.606731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.606893 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.606979 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.607147 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.607228 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:07:40.607331 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.600326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.22.177.2:46787)
0811 10:07:40.601428 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.601438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.601524 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.601693 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.601694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.601886 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:40.601935 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.601936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.602602 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.602603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.603110 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.603112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.603514 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.603515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.603593 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.604069 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:40.604081 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:40.604374 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:40.606159 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.606161 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.606161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.606360 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.606576 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.606577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.606577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.606639 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.606977 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:40.606978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:40.607028 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.607120 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.607230 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"thread_start_us":291,"threads_started":1}
WARNING: no policy specified for oryx/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab.
Entry for principal oryx/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab.
I20260811 10:07:40.629448 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:35837
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:40.719991 31819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:40.720214 31819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:40.720242 31819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:40.721464 31819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:40.721501 31819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:40.721524 31819 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:07:40.721544 31819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:40.723183 31819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.177.62:35837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:40.723383 31819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:40.723562 31819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:40.728817 31828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:40.728837 31826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:40.728817 31825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:40.729266 31819 server_base.cc:1034] running on GCE node
I20260811 10:07:40.729490 31819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:40.729758 31819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:40.730906 31819 hybrid_clock.cc:648] HybridClock initialized: now 1786442860730891 us; error 26 us; skew 500 ppm
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:40.733629 31819 init.cc:377] Logged in from keytab as oryx/127.22.177.3@KRBTEST.COM (short username oryx)
I20260811 10:07:40.734081 31819 webserver.cc:460] Webserver started at http://127.22.177.3:44639/ using document root <none> and password file <none>
I20260811 10:07:40.734285 31819 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:40.734326 31819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:40.734414 31819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:40.735499 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "9986f78a5bf0425db793f4a15b6ee79b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "a120fd77afe3095aa4c435ebfa26e4a9"
server_key_iv: "858e2b35f8792a5b57bdc265e2943fdf"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.735839 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "9986f78a5bf0425db793f4a15b6ee79b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "a120fd77afe3095aa4c435ebfa26e4a9"
server_key_iv: "858e2b35f8792a5b57bdc265e2943fdf"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.737195 31819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:40.738099 31835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.738433 31819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:40.738534 31819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "9986f78a5bf0425db793f4a15b6ee79b"
format_stamp: "Formatted at 2026-08-11 10:07:40 on dist-test-slave-g350"
server_key: "a120fd77afe3095aa4c435ebfa26e4a9"
server_key_iv: "858e2b35f8792a5b57bdc265e2943fdf"
server_key_version: "encryptionkey@0"
I20260811 10:07:40.738621 31819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:40.753265 31819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:40.755914 31819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:40.756083 31819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:40.756647 31819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:40.756717 31819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.756762 31819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:40.756798 31819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.764019 31819 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:45905
I20260811 10:07:40.764088 31958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:45905 every 8 connection(s)
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:40.764439 31819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:40.765823 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 31819
I20260811 10:07:40.765910 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:40.766121 23236 external_mini_cluster.cc:1428] Setting key 8b0ad75d85c923708eee1fc1d00cce83
I20260811 10:07:40.767488 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.757181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:58873 (local address 127.22.177.62:35837)
0811 10:07:40.757298 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.757300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.757497 (+ 197us) server_negotiation.cc:407] Connection header received
0811 10:07:40.758154 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.758156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.758172 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.758242 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.758909 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.759760 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.761332 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.761705 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.761784 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.764732 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.764734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.764734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.764744 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.766068 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.766490 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.766492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.766493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.766553 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.766818 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.766819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.766819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.766945 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.767020 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.767313 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.767397 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:07:40.767676 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.757175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.22.177.3:58873)
0811 10:07:40.757360 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.757407 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.758011 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.758333 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.758338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.758604 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:40.758808 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.758813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.759853 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.759854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.761227 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.761231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.761832 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.761833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.761993 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.762557 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:40.762567 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:40.764621 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:40.766149 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.766157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.766159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.766409 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.766636 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.766636 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.766637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.766749 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.767022 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:40.767030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:40.767170 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.767288 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.767447 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260811 10:07:40.768146 31959 heartbeater.cc:347] Connected to a master server at 127.22.177.62:35837
I20260811 10:07:40.768239 31959 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:40.768424 31959 heartbeater.cc:511] Master 127.22.177.62:35837 requested a full tablet report, sending...
I20260811 10:07:40.768831 31433 ts_manager.cc:194] Registered new tserver with Master: 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:40.769327 31433 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:58873
I20260811 10:07:40.775009 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.769238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:50793 (local address 127.22.177.62:35837)
0811 10:07:40.769278 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.769280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.769351 (+ 71us) server_negotiation.cc:407] Connection header received
0811 10:07:40.769439 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.769440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.769450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.769500 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.769880 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.770352 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.771139 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.771343 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.771417 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.772399 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.772400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.772401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.772407 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.773877 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.774227 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.774228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.774229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.774290 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.774500 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.774501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.774501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.774610 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.774691 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.774843 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.774918 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:07:40.775025 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.769176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.22.177.3:50793)
0811 10:07:40.769297 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.769307 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.769367 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.769502 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.769503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.769734 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:40.769786 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.769788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.770451 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.770452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.771049 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.771050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.771416 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.771417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.771503 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.771985 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:40.771990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:40.772291 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:40.773974 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.773975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.773975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.774154 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.774365 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:40.774365 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:40.774366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:40.774431 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:40.774682 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:40.774683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:40.774730 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.774818 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.774920 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260811 10:07:40.777835 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:07:40.801116 31490 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:40.795270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.22.177.62:35837)
0811 10:07:40.795342 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.795344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.795375 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:07:40.795401 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.795402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.795411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.795457 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.796078 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.796597 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.797379 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.797560 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.797700 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.801012 (+ 3312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39036: BlockingRecv error: recv got EOF from 127.0.0.1:39036 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 10:07:40 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:40.809777 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.801943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.22.177.62:35837)
0811 10:07:40.802026 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.802028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.802038 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:40.802133 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.802134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.802144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.802254 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:40.802746 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.803226 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.803957 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.804160 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.804243 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.806755 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:40.806757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:40.806757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:40.806765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:40.808428 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.808855 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.808857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.808857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.808926 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:40.809208 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:40.809209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:40.809210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:40.809339 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:40.809430 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.809534 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.809706 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 10:07:40.811695 31433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:40.813277 31433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:40.820947 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.816971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.22.177.1:46523)
0811 10:07:40.817121 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.817125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.817137 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:40.817180 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.817182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.817198 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.817515 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:40.817578 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.818330 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.819757 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.820466 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.820537 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.820626 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.820771 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.820839 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260811 10:07:40.820935 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.816489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:46523 (local address 127.0.0.1:41920)
0811 10:07:40.816630 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.816640 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.816682 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.817366 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.817367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.817369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:40.817428 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.817430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.818427 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.818429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.819672 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.819675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.820563 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.820565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.820652 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.820669 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.820758 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.820830 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260811 10:07:40.821296 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.816313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45905 (local address 127.0.0.1:42458)
0811 10:07:40.816592 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.816605 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.816664 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.817197 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.817200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.817203 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:40.817288 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.817290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.818774 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.818775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.820047 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.820049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.820854 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.820855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.820950 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.820955 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.821176 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.821224 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":106,"threads_started":1}
I20260811 10:07:40.821678 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.816733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.22.177.3:45905)
0811 10:07:40.816956 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.816961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.816971 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:40.817006 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.817007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.817052 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.817425 (+ 373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:40.817495 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.818664 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.820134 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.820752 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.820833 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.820907 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.821489 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.821544 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":104,"threads_started":1}
I20260811 10:07:40.821689 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.816219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39677 (local address 127.0.0.1:58350)
0811 10:07:40.816479 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.816528 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.816603 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.817408 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.817409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.817410 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:40.817440 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.817441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.818805 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.818806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.819555 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.819557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.821182 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.821183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.821249 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.821254 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.821555 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.821623 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":114,"threads_started":1}
I20260811 10:07:40.821794 31581 tablet_service.cc:1467] Processing CreateTablet for tablet 836b631ae3c148f19e8ba06252cf6c43 (DEFAULT_TABLE table=test-table [id=00908a1ca9104d28a57cc4a8084d3fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:40.821992 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.816404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.22.177.2:39677)
0811 10:07:40.817193 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.817196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.817206 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:40.817244 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.817246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.817260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.817334 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:40.817532 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.818714 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.819659 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.820455 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.820537 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.820621 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.821831 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.821893 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":89,"threads_started":1}
I20260811 10:07:40.822263 31581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836b631ae3c148f19e8ba06252cf6c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.822623 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 836b631ae3c148f19e8ba06252cf6c43 (DEFAULT_TABLE table=test-table [id=00908a1ca9104d28a57cc4a8084d3fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:40.823009 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836b631ae3c148f19e8ba06252cf6c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.826085 31989 tablet_bootstrap.cc:492] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Bootstrap starting.
I20260811 10:07:40.826442 31990 tablet_bootstrap.cc:492] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Bootstrap starting.
I20260811 10:07:40.826723 31889 tablet_service.cc:1467] Processing CreateTablet for tablet 836b631ae3c148f19e8ba06252cf6c43 (DEFAULT_TABLE table=test-table [id=00908a1ca9104d28a57cc4a8084d3fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:40.826875 31989 tablet_bootstrap.cc:654] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.827072 31889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836b631ae3c148f19e8ba06252cf6c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.827080 31990 tablet_bootstrap.cc:654] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.827239 31989 log.cc:824] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:40.827399 31990 log.cc:824] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:40.828194 31990 tablet_bootstrap.cc:492] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: No bootstrap required, opened a new log
I20260811 10:07:40.828274 31990 ts_tablet_manager.cc:1397] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:40.830363 31989 tablet_bootstrap.cc:492] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: No bootstrap required, opened a new log
I20260811 10:07:40.830457 31989 ts_tablet_manager.cc:1397] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 10:07:40.830444 31990 raft_consensus.cc:348] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.830631 31990 raft_consensus.cc:374] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.830688 31990 raft_consensus.cc:729] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f02ec8d04483b73b2fdc15ff8f9b, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.830824 31990 consensus_queue.cc:260] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.831071 31990 ts_tablet_manager.cc:1428] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:40.831208 31808 heartbeater.cc:503] Master 127.22.177.62:35837 was elected leader, sending a full tablet report...
I20260811 10:07:40.832600 31989 raft_consensus.cc:348] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.832804 31989 raft_consensus.cc:374] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.832854 31989 raft_consensus.cc:729] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.832970 31989 consensus_queue.cc:260] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.833245 31989 ts_tablet_manager.cc:1428] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:07:40.833328 31657 heartbeater.cc:503] Master 127.22.177.62:35837 was elected leader, sending a full tablet report...
I20260811 10:07:40.833565 31993 tablet_bootstrap.cc:492] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Bootstrap starting.
I20260811 10:07:40.834272 31993 tablet_bootstrap.cc:654] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.834654 31993 log.cc:824] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:40.835623 31993 tablet_bootstrap.cc:492] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: No bootstrap required, opened a new log
I20260811 10:07:40.835683 31993 ts_tablet_manager.cc:1397] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:40.837136 31993 raft_consensus.cc:348] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.837266 31993 raft_consensus.cc:374] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.837296 31993 raft_consensus.cc:729] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.837411 31993 consensus_queue.cc:260] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.837708 31993 ts_tablet_manager.cc:1428] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:07:40.837777 31959 heartbeater.cc:503] Master 127.22.177.62:35837 was elected leader, sending a full tablet report...
W20260811 10:07:40.846287 31810 tablet.cc:2369] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:40.909386 31995 raft_consensus.cc:482] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:40.909525 31995 raft_consensus.cc:504] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.909971 31995 leader_election.cc:288] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677), 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:40.913460 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.910209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45905 (local address 127.22.177.1:57329)
0811 10:07:40.910277 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.910289 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.910355 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.910678 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.910679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.910681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:40.910760 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.910762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.911524 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.911525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.912364 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.912367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.913137 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.913138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.913235 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.913239 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.913323 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.913374 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":2}
I20260811 10:07:40.913504 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.910466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:57329 (local address 127.22.177.3:45905)
0811 10:07:40.910512 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.910514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.910522 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:40.910551 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.910552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.910562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.910672 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:40.910845 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.911445 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.912518 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.913026 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.913085 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.913093 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.913375 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.913427 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:07:40.913770 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.910213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39677 (local address 127.22.177.1:58809)
0811 10:07:40.910403 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:40.910412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:40.910457 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:40.910862 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:40.910862 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:40.910864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:40.910930 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.910931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.911743 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.911744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.912715 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:40.912718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.913410 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.913412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:40.913487 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.913491 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:40.913552 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:07:40.913627 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":73,"thread_start_us":80,"threads_started":1}
I20260811 10:07:40.913818 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.910515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:58809 (local address 127.22.177.2:39677)
0811 10:07:40.910572 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.910575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.910583 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:40.910614 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.910614 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.910624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.910686 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:40.911021 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.911629 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.912823 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.913305 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.913372 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.913380 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.913690 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.913740 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:07:40.913940 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836b631ae3c148f19e8ba06252cf6c43" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9986f78a5bf0425db793f4a15b6ee79b" is_pre_election: true
I20260811 10:07:40.914160 31912 raft_consensus.cc:2454] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 0.
I20260811 10:07:40.914335 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836b631ae3c148f19e8ba06252cf6c43" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" is_pre_election: true
I20260811 10:07:40.914467 31529 leader_election.cc:302] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:40.914508 31758 raft_consensus.cc:2454] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 0.
I20260811 10:07:40.914615 31995 raft_consensus.cc:2781] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:40.914726 31995 raft_consensus.cc:482] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:40.914754 31995 raft_consensus.cc:3037] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:40.915480 31995 raft_consensus.cc:504] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.915630 31995 leader_election.cc:288] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677), 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:40.915820 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836b631ae3c148f19e8ba06252cf6c43" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
I20260811 10:07:40.915824 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836b631ae3c148f19e8ba06252cf6c43" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9986f78a5bf0425db793f4a15b6ee79b"
I20260811 10:07:40.915921 31912 raft_consensus.cc:3037] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:40.915916 31758 raft_consensus.cc:3037] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:40.916652 31912 raft_consensus.cc:2454] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 1.
I20260811 10:07:40.916718 31758 raft_consensus.cc:2454] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 1.
I20260811 10:07:40.916872 31529 leader_election.cc:302] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:40.917001 31995 raft_consensus.cc:2781] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:40.917208 31995 raft_consensus.cc:686] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Running, Role: LEADER
I20260811 10:07:40.917325 31995 consensus_queue.cc:237] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } }
I20260811 10:07:40.918334 31431 catalog_manager.cc:5495] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1). New cstate: current_term: 1 leader_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } health_report { overall_health: HEALTHY } } }
W20260811 10:07:40.928197 31658 tablet.cc:2369] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:40.938824 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:40.936237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.22.177.1:46523)
0811 10:07:40.936318 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:40.936320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:40.936338 (+ 18us) server_negotiation.cc:407] Connection header received
0811 10:07:40.936384 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:40.936385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:40.936403 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:40.936467 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:40.936648 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.937119 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.937990 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:40.938193 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:40.938265 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:40.938413 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:40.938515 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:40.938578 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:40.938676 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 10:07:40.938697 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 10:07:40.940763 31912 raft_consensus.cc:1261] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Refusing update from remote peer 17cf266dba5d4e4abd29e32b8f501ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:40.940764 31758 raft_consensus.cc:1261] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Refusing update from remote peer 17cf266dba5d4e4abd29e32b8f501ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:40.941133 31995 consensus_queue.cc:1035] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:40.941321 32000 consensus_queue.cc:1035] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:40.945899 32005 mvcc.cc:205] Tried to move back new op lower bound from 7317269958410354688 to 7317269958318075904. Current Snapshot: MvccSnapshot[applied={T|T < 7317269958410354688 or (T in {7317269958410354688})}]
I20260811 10:07:40.946147 31432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:40.946251 31432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:40.947057 31432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Sending DeleteTablet for 3 replicas of tablet 836b631ae3c148f19e8ba06252cf6c43
I20260811 10:07:40.947377 31889 tablet_service.cc:1514] Processing DeleteTablet for tablet 836b631ae3c148f19e8ba06252cf6c43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:40 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:42458
I20260811 10:07:40.947487 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet 836b631ae3c148f19e8ba06252cf6c43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:40 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:58350
I20260811 10:07:40.947615 32013 tablet_replica.cc:330] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: stopping tablet replica
I20260811 10:07:40.947674 31581 tablet_service.cc:1514] Processing DeleteTablet for tablet 836b631ae3c148f19e8ba06252cf6c43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:40 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41920
I20260811 10:07:40.947716 32013 raft_consensus.cc:2229] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:40.948053 32016 tablet_replica.cc:330] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: stopping tablet replica
I20260811 10:07:40.948107 32013 raft_consensus.cc:2258] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:40.948146 32016 raft_consensus.cc:2229] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:40.948150 31432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:40.948315 31432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:40.948400 32014 tablet_replica.cc:330] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: stopping tablet replica
I20260811 10:07:40.948493 32014 raft_consensus.cc:2229] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:40.948768 32013 ts_tablet_manager.cc:1905] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:40.948944 32014 raft_consensus.cc:2258] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:40.948233 32016 raft_consensus.cc:2258] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:40.949677 32016 ts_tablet_manager.cc:1905] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:40.950078 31889 tablet_service.cc:1467] Processing CreateTablet for tablet c61e5eac928a44f1a82c2ed1b8a2fef7 (DEFAULT_TABLE table=test-table [id=1a5c387a8da546c5959dafbe70bc7040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:40.950147 31737 tablet_service.cc:1467] Processing CreateTablet for tablet c61e5eac928a44f1a82c2ed1b8a2fef7 (DEFAULT_TABLE table=test-table [id=1a5c387a8da546c5959dafbe70bc7040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:40.950206 31889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c61e5eac928a44f1a82c2ed1b8a2fef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.950291 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c61e5eac928a44f1a82c2ed1b8a2fef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.950809 32014 ts_tablet_manager.cc:1905] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:40.951793 31993 tablet_bootstrap.cc:492] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Bootstrap starting.
I20260811 10:07:40.951793 31581 tablet_service.cc:1467] Processing CreateTablet for tablet c61e5eac928a44f1a82c2ed1b8a2fef7 (DEFAULT_TABLE table=test-table [id=1a5c387a8da546c5959dafbe70bc7040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:40.951921 31581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c61e5eac928a44f1a82c2ed1b8a2fef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:40.951956 32013 ts_tablet_manager.cc:1918] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:40.952023 32013 log.cc:1196] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/836b631ae3c148f19e8ba06252cf6c43
I20260811 10:07:40.952271 31993 tablet_bootstrap.cc:654] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.952309 32013 ts_tablet_manager.cc:1939] T 836b631ae3c148f19e8ba06252cf6c43 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting consensus metadata
I20260811 10:07:40.952751 31417 catalog_manager.cc:4841] TS 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905): tablet 836b631ae3c148f19e8ba06252cf6c43 (table test-table [id=00908a1ca9104d28a57cc4a8084d3fe4]) successfully deleted
I20260811 10:07:40.952929 31993 tablet_bootstrap.cc:492] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: No bootstrap required, opened a new log
I20260811 10:07:40.952971 31993 ts_tablet_manager.cc:1397] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:40.953123 31993 raft_consensus.cc:348] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:40.953205 31993 raft_consensus.cc:374] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.953239 31993 raft_consensus.cc:729] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.953292 31993 consensus_queue.cc:260] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:40.953385 31993 ts_tablet_manager.cc:1428] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:40.953480 31989 tablet_bootstrap.cc:492] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Bootstrap starting.
I20260811 10:07:40.953522 31990 tablet_bootstrap.cc:492] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Bootstrap starting.
I20260811 10:07:40.953969 31989 tablet_bootstrap.cc:654] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.953969 31990 tablet_bootstrap.cc:654] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:40.954833 31989 tablet_bootstrap.cc:492] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: No bootstrap required, opened a new log
I20260811 10:07:40.954895 31989 ts_tablet_manager.cc:1397] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:40.955049 31989 raft_consensus.cc:348] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:40.955144 31989 raft_consensus.cc:374] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.955168 31989 raft_consensus.cc:729] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.955236 31990 tablet_bootstrap.cc:492] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: No bootstrap required, opened a new log
I20260811 10:07:40.955219 31989 consensus_queue.cc:260] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:40.955286 31990 ts_tablet_manager.cc:1397] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:40.955317 31989 ts_tablet_manager.cc:1428] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:40.955452 31990 raft_consensus.cc:348] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:40.955528 31990 raft_consensus.cc:374] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:40.955549 31990 raft_consensus.cc:729] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f02ec8d04483b73b2fdc15ff8f9b, State: Initialized, Role: FOLLOWER
I20260811 10:07:40.955591 31990 consensus_queue.cc:260] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:40.955682 31990 ts_tablet_manager.cc:1428] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:40.958593 32016 ts_tablet_manager.cc:1918] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:40.958667 32016 log.cc:1196] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/836b631ae3c148f19e8ba06252cf6c43
I20260811 10:07:40.958936 32016 ts_tablet_manager.cc:1939] T 836b631ae3c148f19e8ba06252cf6c43 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting consensus metadata
I20260811 10:07:40.959563 31420 catalog_manager.cc:4841] TS 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523): tablet 836b631ae3c148f19e8ba06252cf6c43 (table test-table [id=00908a1ca9104d28a57cc4a8084d3fe4]) successfully deleted
I20260811 10:07:40.959961 32014 ts_tablet_manager.cc:1918] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:40.960065 32014 log.cc:1196] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/836b631ae3c148f19e8ba06252cf6c43
I20260811 10:07:40.960357 32014 ts_tablet_manager.cc:1939] T 836b631ae3c148f19e8ba06252cf6c43 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting consensus metadata
I20260811 10:07:40.960747 31420 catalog_manager.cc:4841] TS 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677): tablet 836b631ae3c148f19e8ba06252cf6c43 (table test-table [id=00908a1ca9104d28a57cc4a8084d3fe4]) successfully deleted
W20260811 10:07:41.015153 31960 tablet.cc:2369] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:07:41.124310 31995 raft_consensus.cc:482] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:41.124428 31995 raft_consensus.cc:504] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.124624 31995 leader_election.cc:288] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677), 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:41.124944 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61e5eac928a44f1a82c2ed1b8a2fef7" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" is_pre_election: true
I20260811 10:07:41.124990 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61e5eac928a44f1a82c2ed1b8a2fef7" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9986f78a5bf0425db793f4a15b6ee79b" is_pre_election: true
I20260811 10:07:41.125082 31758 raft_consensus.cc:2454] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 0.
I20260811 10:07:41.125077 31912 raft_consensus.cc:2454] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 0.
I20260811 10:07:41.125281 31532 leader_election.cc:302] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 4f51f02ec8d04483b73b2fdc15ff8f9b; no voters:
I20260811 10:07:41.125415 31995 raft_consensus.cc:2781] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:41.125476 31995 raft_consensus.cc:482] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:41.125494 31995 raft_consensus.cc:3037] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.126241 31995 raft_consensus.cc:504] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.126386 31995 leader_election.cc:288] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677), 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:41.126557 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61e5eac928a44f1a82c2ed1b8a2fef7" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
I20260811 10:07:41.126556 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61e5eac928a44f1a82c2ed1b8a2fef7" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9986f78a5bf0425db793f4a15b6ee79b"
I20260811 10:07:41.126646 31758 raft_consensus.cc:3037] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.126646 31912 raft_consensus.cc:3037] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.127476 31912 raft_consensus.cc:2454] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 1.
I20260811 10:07:41.127516 31758 raft_consensus.cc:2454] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 1.
I20260811 10:07:41.127676 31532 leader_election.cc:302] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 4f51f02ec8d04483b73b2fdc15ff8f9b; no voters:
I20260811 10:07:41.127799 31995 raft_consensus.cc:2781] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:41.127869 31995 raft_consensus.cc:686] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Running, Role: LEADER
I20260811 10:07:41.127940 31995 consensus_queue.cc:237] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.128670 31433 catalog_manager.cc:5495] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1). New cstate: current_term: 1 leader_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } health_report { overall_health: UNKNOWN } } }
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442860, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.62@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:41.161789 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.154532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:35837 (local address 127.0.0.1:39058)
0811 10:07:41.154727 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.154737 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.154822 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.154957 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.154957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.155284 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:41.155334 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.155336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.156011 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.156012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.156549 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.156550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.156921 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.156922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.156977 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.157552 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:41.157556 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:41.159060 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:41.160793 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:41.160800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:41.160802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:41.160991 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:41.161167 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:41.161167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:41.161168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:41.161225 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:41.161427 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:41.161433 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:41.161474 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.161529 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.161674 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260811 10:07:41.161845 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.154575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39058 (local address 127.22.177.62:35837)
0811 10:07:41.154656 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.154659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.154835 (+ 176us) server_negotiation.cc:407] Connection header received
0811 10:07:41.154866 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.154867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.154878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.154938 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:41.155413 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.155936 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.156630 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.156826 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.156884 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.159143 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:41.159144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:41.159144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:41.159205 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:41.160714 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:41.161056 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:41.161057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:41.161057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:41.161105 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:41.161265 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:41.161266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:41.161266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:41.161367 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:41.161428 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.161666 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.161742 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:07:41.164014 31433 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:39058:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 10:07:41.164232 31433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:41.165745 31581 tablet_service.cc:1467] Processing CreateTablet for tablet a35a4c06cd1d4fd697c8b30c7b6ca1d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91db51dab7104bb18a1f0e3d3032815a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:41.165745 31889 tablet_service.cc:1467] Processing CreateTablet for tablet a35a4c06cd1d4fd697c8b30c7b6ca1d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91db51dab7104bb18a1f0e3d3032815a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:41.165918 31581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35a4c06cd1d4fd697c8b30c7b6ca1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.165923 31889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35a4c06cd1d4fd697c8b30c7b6ca1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.165879 31737 tablet_service.cc:1467] Processing CreateTablet for tablet a35a4c06cd1d4fd697c8b30c7b6ca1d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91db51dab7104bb18a1f0e3d3032815a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:07:41.166044 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35a4c06cd1d4fd697c8b30c7b6ca1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.167491 31989 tablet_bootstrap.cc:492] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: Bootstrap starting.
I20260811 10:07:41.167500 31993 tablet_bootstrap.cc:492] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: Bootstrap starting.
I20260811 10:07:41.167886 31989 tablet_bootstrap.cc:654] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.167937 31993 tablet_bootstrap.cc:654] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.168234 31990 tablet_bootstrap.cc:492] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Bootstrap starting.
I20260811 10:07:41.168473 31989 tablet_bootstrap.cc:492] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: No bootstrap required, opened a new log
I20260811 10:07:41.168521 31989 ts_tablet_manager.cc:1397] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:41.168568 31993 tablet_bootstrap.cc:492] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: No bootstrap required, opened a new log
I20260811 10:07:41.168637 31990 tablet_bootstrap.cc:654] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.168653 31993 ts_tablet_manager.cc:1397] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:41.168646 31989 raft_consensus.cc:348] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.168727 31989 raft_consensus.cc:374] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.168749 31989 raft_consensus.cc:729] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.168792 31989 consensus_queue.cc:260] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.168787 31993 raft_consensus.cc:348] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.168867 31993 raft_consensus.cc:374] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.168881 31989 ts_tablet_manager.cc:1428] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:41.168893 31993 raft_consensus.cc:729] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.168937 31993 consensus_queue.cc:260] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.169023 31993 ts_tablet_manager.cc:1428] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:41.168992 31995 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.169060 31995 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.169173 31997 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.169255 31997 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.169341 31990 tablet_bootstrap.cc:492] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: No bootstrap required, opened a new log
I20260811 10:07:41.169375 31990 ts_tablet_manager.cc:1397] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:41.169525 31990 raft_consensus.cc:348] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.169589 31990 raft_consensus.cc:374] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.169641 31990 raft_consensus.cc:729] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f02ec8d04483b73b2fdc15ff8f9b, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.169684 31990 consensus_queue.cc:260] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.169790 31990 ts_tablet_manager.cc:1428] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:07:41.169929 31994 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.170010 31994 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.349642 31997 raft_consensus.cc:482] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:41.349807 31997 raft_consensus.cc:504] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.350265 31997 leader_election.cc:288] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523), 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677)
I20260811 10:07:41.353493 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.350518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:49495 (local address 127.22.177.1:46523)
0811 10:07:41.350596 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.350599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.350704 (+ 105us) server_negotiation.cc:407] Connection header received
0811 10:07:41.350857 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.350859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.350875 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.350931 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:41.351166 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.351651 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.352745 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.353099 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.353157 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.353165 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.353359 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.353410 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:07:41.353518 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.350734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:42673 (local address 127.22.177.2:39677)
0811 10:07:41.350784 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.350787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.350821 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:07:41.350932 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.350933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.350964 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.351009 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:07:41.351179 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.351735 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.352703 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.353120 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.353230 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.353239 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.353407 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.353450 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:07:41.353516 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.350641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39677 (local address 127.22.177.3:42673)
0811 10:07:41.350762 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.350773 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.350837 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.351025 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.351026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.351028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:41.351094 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.351096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.351828 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.351829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.352594 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.352596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.353248 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.353249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.353304 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.353308 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.353383 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.353423 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260811 10:07:41.353525 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.350423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:46523 (local address 127.22.177.3:49495)
0811 10:07:41.350575 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.350594 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.350729 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.350968 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.350969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.350972 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:07:41.351074 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.351076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.351745 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.351746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.352610 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.352613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.353191 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.353193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.353280 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.353284 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.353343 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.353411 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":23,"thread_start_us":86,"threads_started":1}
I20260811 10:07:41.354082 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35a4c06cd1d4fd697c8b30c7b6ca1d2" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" is_pre_election: true
I20260811 10:07:41.354231 31758 raft_consensus.cc:2454] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 0.
I20260811 10:07:41.354377 31603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35a4c06cd1d4fd697c8b30c7b6ca1d2" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" is_pre_election: true
I20260811 10:07:41.354437 31839 leader_election.cc:302] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51f02ec8d04483b73b2fdc15ff8f9b, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:41.354537 31603 raft_consensus.cc:2454] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 0.
I20260811 10:07:41.354590 31997 raft_consensus.cc:2781] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:41.354640 31997 raft_consensus.cc:482] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:41.354669 31997 raft_consensus.cc:3037] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.355408 31997 raft_consensus.cc:504] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.355556 31997 leader_election.cc:288] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 election: Requested vote from peers 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523), 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677)
I20260811 10:07:41.355705 31603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35a4c06cd1d4fd697c8b30c7b6ca1d2" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cf266dba5d4e4abd29e32b8f501ff4"
I20260811 10:07:41.355787 31603 raft_consensus.cc:3037] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.355763 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35a4c06cd1d4fd697c8b30c7b6ca1d2" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
I20260811 10:07:41.355840 31758 raft_consensus.cc:3037] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.356600 31758 raft_consensus.cc:2454] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 1.
I20260811 10:07:41.356619 31603 raft_consensus.cc:2454] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 1.
I20260811 10:07:41.356853 31839 leader_election.cc:302] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51f02ec8d04483b73b2fdc15ff8f9b, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:41.356945 31997 raft_consensus.cc:2781] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:41.357061 31997 raft_consensus.cc:686] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 LEADER]: Becoming Leader. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Running, Role: LEADER
I20260811 10:07:41.357147 31997 consensus_queue.cc:237] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.357471 32035 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: TxnStatusTablet state changed. Reason: New leader 9986f78a5bf0425db793f4a15b6ee79b. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.357580 32035 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:41.357828 32037 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 10:07:41.357956 32037 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 10:07:41.358002 31433 catalog_manager.cc:5495] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3). New cstate: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260811 10:07:41.405071 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.401451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45905 (local address 127.0.0.1:42470)
0811 10:07:41.401546 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.401557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.401639 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.401852 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.401854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.401855 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:41.401917 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.401921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.402967 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.402969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.403804 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.403808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.404344 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.404345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.404447 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.404469 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.404820 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.404832 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.404912 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.404973 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 10:07:41.405071 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.401509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.22.177.3:45905)
0811 10:07:41.401557 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.401560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.401655 (+ 95us) server_negotiation.cc:407] Connection header received
0811 10:07:41.401730 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.401731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.401755 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.401841 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:41.401995 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.402881 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.403907 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.404220 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.404313 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.404577 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.404736 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.404824 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.404923 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.404952 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:07:41.406600 31758 raft_consensus.cc:1261] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Refusing update from remote peer 9986f78a5bf0425db793f4a15b6ee79b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.406703 31603 raft_consensus.cc:1261] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Refusing update from remote peer 9986f78a5bf0425db793f4a15b6ee79b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.406908 31997 consensus_queue.cc:1035] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.407090 31997 consensus_queue.cc:1035] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.407614 31994 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: TxnStatusTablet state changed. Reason: New leader 9986f78a5bf0425db793f4a15b6ee79b. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.407719 31994 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.407835 31995 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: TxnStatusTablet state changed. Reason: New leader 9986f78a5bf0425db793f4a15b6ee79b. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.407936 31995 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.409086 31997 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.409180 31997 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:41.409199 32004 mvcc.cc:205] Tried to move back new op lower bound from 7317269960318681088 to 7317269960119545856. Current Snapshot: MvccSnapshot[applied={T|T < 7317269960318681088}]
I20260811 10:07:41.409235 31997 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.409299 31997 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 9986f78a5bf0425db793f4a15b6ee79b: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:07:41.409489 31994 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.409576 31994 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 4f51f02ec8d04483b73b2fdc15ff8f9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.410027 31995 tablet_replica.cc:439] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } } }
I20260811 10:07:41.410128 31995 tablet_replica.cc:442] T a35a4c06cd1d4fd697c8b30c7b6ca1d2 P 17cf266dba5d4e4abd29e32b8f501ff4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:07:41.417495 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.414237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45905 (local address 127.22.177.1:49291)
0811 10:07:41.414361 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.414374 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.414441 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.414589 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.414590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.414591 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:41.414643 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.414645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.415507 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.415508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.416444 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.416448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.416965 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.416966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.417038 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.417058 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.417294 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.417297 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.417347 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.417404 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":59,"threads_started":1}
I20260811 10:07:41.417495 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.414305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:49291 (local address 127.22.177.3:45905)
0811 10:07:41.414353 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.414355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.414421 (+ 66us) server_negotiation.cc:407] Connection header received
0811 10:07:41.414509 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.414510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.414522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.414585 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:41.414719 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.415415 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.416556 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.416862 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.416904 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.417164 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.417210 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.417281 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.417353 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.417396 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:07:41.420288 31912 raft_consensus.cc:1261] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Refusing update from remote peer 17cf266dba5d4e4abd29e32b8f501ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.420311 31758 raft_consensus.cc:1261] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Refusing update from remote peer 17cf266dba5d4e4abd29e32b8f501ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.420514 32000 consensus_queue.cc:1035] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.420620 31995 consensus_queue.cc:1035] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.422679 32003 mvcc.cc:205] Tried to move back new op lower bound from 7317269960375525376 to 7317269959180591104. Current Snapshot: MvccSnapshot[applied={T|T < 7317269960375525376}]
I20260811 10:07:41.432921 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.429122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:46005 (local address 127.22.177.1:46523)
0811 10:07:41.429159 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.429161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.429170 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:41.429259 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.429260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.429269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.429323 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:41.429544 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.430338 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.431966 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.432285 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.432355 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.432590 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.432674 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.432706 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.432819 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.432845 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:07:41.433208 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.428944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:46523 (local address 127.22.177.3:46005)
0811 10:07:41.429110 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.429123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.429179 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.429374 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.429375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.429377 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:41.429452 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.429455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.430443 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.430443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.431853 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.431857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.432396 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.432397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.432475 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.432495 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.432755 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.432757 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.433066 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.433119 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260811 10:07:41.442518 31433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:41.442639 31433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:41.443348 31433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Sending DeleteTablet for 3 replicas of tablet c61e5eac928a44f1a82c2ed1b8a2fef7
I20260811 10:07:41.443609 31889 tablet_service.cc:1514] Processing DeleteTablet for tablet c61e5eac928a44f1a82c2ed1b8a2fef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:42458
I20260811 10:07:41.443732 32013 tablet_replica.cc:330] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: stopping tablet replica
I20260811 10:07:41.443713 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet c61e5eac928a44f1a82c2ed1b8a2fef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:58350
I20260811 10:07:41.443786 32013 raft_consensus.cc:2229] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:41.443826 32013 raft_consensus.cc:2258] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.443863 31581 tablet_service.cc:1514] Processing DeleteTablet for tablet c61e5eac928a44f1a82c2ed1b8a2fef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41920
I20260811 10:07:41.443894 32014 tablet_replica.cc:330] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: stopping tablet replica
I20260811 10:07:41.443953 32014 raft_consensus.cc:2229] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:41.443955 32016 tablet_replica.cc:330] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: stopping tablet replica
I20260811 10:07:41.443998 32014 raft_consensus.cc:2258] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.444007 32016 raft_consensus.cc:2229] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:41.444064 32016 raft_consensus.cc:2258] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.444386 32016 ts_tablet_manager.cc:1905] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.444511 32014 ts_tablet_manager.cc:1905] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.446278 32013 ts_tablet_manager.cc:1905] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.446342 32014 ts_tablet_manager.cc:1918] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:41.446391 32014 log.cc:1196] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/c61e5eac928a44f1a82c2ed1b8a2fef7
I20260811 10:07:41.446468 32016 ts_tablet_manager.cc:1918] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:41.446530 32016 log.cc:1196] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/c61e5eac928a44f1a82c2ed1b8a2fef7
I20260811 10:07:41.446609 32014 ts_tablet_manager.cc:1939] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting consensus metadata
I20260811 10:07:41.446808 32016 ts_tablet_manager.cc:1939] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting consensus metadata
I20260811 10:07:41.446943 31420 catalog_manager.cc:4841] TS 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677): tablet c61e5eac928a44f1a82c2ed1b8a2fef7 (table test-table [id=1a5c387a8da546c5959dafbe70bc7040]) successfully deleted
I20260811 10:07:41.447131 31420 catalog_manager.cc:4841] TS 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523): tablet c61e5eac928a44f1a82c2ed1b8a2fef7 (table test-table [id=1a5c387a8da546c5959dafbe70bc7040]) successfully deleted
I20260811 10:07:41.448530 32013 ts_tablet_manager.cc:1918] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:41.448607 32013 log.cc:1196] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/c61e5eac928a44f1a82c2ed1b8a2fef7
I20260811 10:07:41.448846 32013 ts_tablet_manager.cc:1939] T c61e5eac928a44f1a82c2ed1b8a2fef7 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting consensus metadata
I20260811 10:07:41.449416 31417 catalog_manager.cc:4841] TS 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905): tablet c61e5eac928a44f1a82c2ed1b8a2fef7 (table test-table [id=1a5c387a8da546c5959dafbe70bc7040]) successfully deleted
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[31385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:07:41.476015 31490 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:41.468988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.22.177.62:35837)
0811 10:07:41.469049 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.469052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.469195 (+ 143us) server_negotiation.cc:407] Connection header received
0811 10:07:41.469260 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.469261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.469275 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.469338 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:41.470076 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.470870 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.471810 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.472104 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.472191 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.475937 (+ 3746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39072: BlockingRecv error: recv got EOF from 127.0.0.1:39072 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[31385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM
I20260811 10:07:41.485674 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.477010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.22.177.62:35837)
0811 10:07:41.477079 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.477082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.477152 (+ 70us) server_negotiation.cc:407] Connection header received
0811 10:07:41.477286 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.477287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.477300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.477386 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:41.477965 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.478721 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.479549 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.479864 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.479942 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.482600 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:41.482602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:41.482602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:41.482611 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:41.484266 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:41.484682 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:41.484683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:41.484683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:41.484757 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:41.485030 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:41.485031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:41.485031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:41.485182 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:41.485292 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.485451 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.485541 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:07:41.487538 31433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:07:41.487736 31433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:41.489251 31581 tablet_service.cc:1467] Processing CreateTablet for tablet 81a5280811db4ad198b8a76ac5d7920b (DEFAULT_TABLE table=test-table [id=85c6dd7651ba48d680a2394b21af26b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:41.489329 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 81a5280811db4ad198b8a76ac5d7920b (DEFAULT_TABLE table=test-table [id=85c6dd7651ba48d680a2394b21af26b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:41.489315 31889 tablet_service.cc:1467] Processing CreateTablet for tablet 81a5280811db4ad198b8a76ac5d7920b (DEFAULT_TABLE table=test-table [id=85c6dd7651ba48d680a2394b21af26b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:41.489409 31581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81a5280811db4ad198b8a76ac5d7920b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.489463 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81a5280811db4ad198b8a76ac5d7920b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.489463 31889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81a5280811db4ad198b8a76ac5d7920b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.490942 31989 tablet_bootstrap.cc:492] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Bootstrap starting.
I20260811 10:07:41.490952 31993 tablet_bootstrap.cc:492] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Bootstrap starting.
I20260811 10:07:41.491264 31989 tablet_bootstrap.cc:654] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.491436 31993 tablet_bootstrap.cc:654] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.491859 31990 tablet_bootstrap.cc:492] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Bootstrap starting.
I20260811 10:07:41.491925 31989 tablet_bootstrap.cc:492] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: No bootstrap required, opened a new log
I20260811 10:07:41.491976 31989 ts_tablet_manager.cc:1397] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:41.492089 31989 raft_consensus.cc:348] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.492172 31989 raft_consensus.cc:374] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.492194 31989 raft_consensus.cc:729] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.492241 31989 consensus_queue.cc:260] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.492332 31989 ts_tablet_manager.cc:1428] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:07:41.492326 31990 tablet_bootstrap.cc:654] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.492937 31993 tablet_bootstrap.cc:492] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: No bootstrap required, opened a new log
I20260811 10:07:41.492996 31993 ts_tablet_manager.cc:1397] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:41.493170 31993 raft_consensus.cc:348] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.493256 31993 raft_consensus.cc:374] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.493292 31993 raft_consensus.cc:729] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.493352 31993 consensus_queue.cc:260] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.493453 31993 ts_tablet_manager.cc:1428] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:41.493947 31990 tablet_bootstrap.cc:492] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: No bootstrap required, opened a new log
I20260811 10:07:41.494025 31990 ts_tablet_manager.cc:1397] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:07:41.494231 31990 raft_consensus.cc:348] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.494305 31990 raft_consensus.cc:374] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.494330 31990 raft_consensus.cc:729] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f02ec8d04483b73b2fdc15ff8f9b, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.494406 31990 consensus_queue.cc:260] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.494505 31990 ts_tablet_manager.cc:1428] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:41.529861 31995 raft_consensus.cc:482] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:41.530004 31995 raft_consensus.cc:504] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.530225 31995 leader_election.cc:288] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677), 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:41.530458 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81a5280811db4ad198b8a76ac5d7920b" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9986f78a5bf0425db793f4a15b6ee79b" is_pre_election: true
I20260811 10:07:41.530505 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81a5280811db4ad198b8a76ac5d7920b" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" is_pre_election: true
I20260811 10:07:41.530591 31912 raft_consensus.cc:2454] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 0.
I20260811 10:07:41.530620 31758 raft_consensus.cc:2454] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 0.
I20260811 10:07:41.530764 31529 leader_election.cc:302] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:41.530897 31995 raft_consensus.cc:2781] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:41.530944 31995 raft_consensus.cc:482] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:41.530992 31995 raft_consensus.cc:3037] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.531922 31995 raft_consensus.cc:504] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.532065 31995 leader_election.cc:288] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677), 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905)
I20260811 10:07:41.532215 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81a5280811db4ad198b8a76ac5d7920b" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9986f78a5bf0425db793f4a15b6ee79b"
I20260811 10:07:41.532260 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81a5280811db4ad198b8a76ac5d7920b" candidate_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
I20260811 10:07:41.532311 31912 raft_consensus.cc:3037] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.532394 31758 raft_consensus.cc:3037] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.533136 31758 raft_consensus.cc:2454] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 1.
I20260811 10:07:41.533136 31912 raft_consensus.cc:2454] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cf266dba5d4e4abd29e32b8f501ff4 in term 1.
I20260811 10:07:41.533350 31532 leader_election.cc:302] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 4f51f02ec8d04483b73b2fdc15ff8f9b; no voters:
I20260811 10:07:41.533442 31995 raft_consensus.cc:2781] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:41.533476 31995 raft_consensus.cc:686] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Running, Role: LEADER
I20260811 10:07:41.533528 31995 consensus_queue.cc:237] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.534317 31433 catalog_manager.cc:5495] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1). New cstate: current_term: 1 leader_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260811 10:07:41.545569 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.542990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.22.177.1:46523)
0811 10:07:41.543037 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.543039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.543048 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:41.543075 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.543076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.543094 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.543150 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:41.543522 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.544011 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.544856 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.545044 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.545113 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.545283 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.545312 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.545359 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.545482 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.545506 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:07:41.546526 31912 raft_consensus.cc:1261] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Refusing update from remote peer 17cf266dba5d4e4abd29e32b8f501ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.546566 31758 raft_consensus.cc:1261] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Refusing update from remote peer 17cf266dba5d4e4abd29e32b8f501ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.546797 31995 consensus_queue.cc:1035] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.546911 32000 consensus_queue.cc:1035] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.549180 31431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:41.549276 31431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:41.550107 31431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Sending DeleteTablet for 3 replicas of tablet 81a5280811db4ad198b8a76ac5d7920b
I20260811 10:07:41.550522 31581 tablet_service.cc:1514] Processing DeleteTablet for tablet 81a5280811db4ad198b8a76ac5d7920b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41920
I20260811 10:07:41.550523 31889 tablet_service.cc:1514] Processing DeleteTablet for tablet 81a5280811db4ad198b8a76ac5d7920b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:42458
I20260811 10:07:41.550542 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet 81a5280811db4ad198b8a76ac5d7920b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:58350
I20260811 10:07:41.550622 32016 tablet_replica.cc:330] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: stopping tablet replica
I20260811 10:07:41.550642 32013 tablet_replica.cc:330] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: stopping tablet replica
I20260811 10:07:41.550647 32014 tablet_replica.cc:330] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: stopping tablet replica
I20260811 10:07:41.550678 32016 raft_consensus.cc:2229] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:41.550750 32016 raft_consensus.cc:2258] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.550691 32014 raft_consensus.cc:2229] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:41.550691 32013 raft_consensus.cc:2229] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:41.550861 32013 raft_consensus.cc:2258] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.550851 32014 raft_consensus.cc:2258] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.551038 32016 ts_tablet_manager.cc:1905] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.551307 31431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 10:07:41.551389 32014 ts_tablet_manager.cc:1905] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 10:07:41.551461 31431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:07:41.552953 31889 tablet_service.cc:1467] Processing CreateTablet for tablet f362f4da55cb41c3b98392458540fd74 (DEFAULT_TABLE table=test-table [id=5247f31846ee4a3c8a31aeb763395ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:41.552996 31581 tablet_service.cc:1467] Processing CreateTablet for tablet f362f4da55cb41c3b98392458540fd74 (DEFAULT_TABLE table=test-table [id=5247f31846ee4a3c8a31aeb763395ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:41.553007 32016 ts_tablet_manager.cc:1918] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:41.553082 32016 log.cc:1196] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/81a5280811db4ad198b8a76ac5d7920b
I20260811 10:07:41.553098 31889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f362f4da55cb41c3b98392458540fd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.553117 31581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f362f4da55cb41c3b98392458540fd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.553320 32016 ts_tablet_manager.cc:1939] T 81a5280811db4ad198b8a76ac5d7920b P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting consensus metadata
I20260811 10:07:41.553648 32014 ts_tablet_manager.cc:1918] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:41.553655 31420 catalog_manager.cc:4841] TS 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523): tablet 81a5280811db4ad198b8a76ac5d7920b (table test-table [id=85c6dd7651ba48d680a2394b21af26b0]) successfully deleted
I20260811 10:07:41.553704 32014 log.cc:1196] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/81a5280811db4ad198b8a76ac5d7920b
I20260811 10:07:41.553958 32014 ts_tablet_manager.cc:1939] T 81a5280811db4ad198b8a76ac5d7920b P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting consensus metadata
I20260811 10:07:41.554277 31989 tablet_bootstrap.cc:492] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: Bootstrap starting.
I20260811 10:07:41.554495 32013 ts_tablet_manager.cc:1905] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.554533 31989 tablet_bootstrap.cc:654] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.554840 31420 catalog_manager.cc:4841] TS 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677): tablet 81a5280811db4ad198b8a76ac5d7920b (table test-table [id=85c6dd7651ba48d680a2394b21af26b0]) successfully deleted
I20260811 10:07:41.555315 31989 tablet_bootstrap.cc:492] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: No bootstrap required, opened a new log
I20260811 10:07:41.555356 31989 ts_tablet_manager.cc:1397] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:07:41.555440 31989 raft_consensus.cc:348] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.555495 31989 raft_consensus.cc:374] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.555508 31989 raft_consensus.cc:729] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cf266dba5d4e4abd29e32b8f501ff4, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.555572 31989 consensus_queue.cc:260] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.555670 31989 ts_tablet_manager.cc:1428] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:41.556017 32013 ts_tablet_manager.cc:1918] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:07:41.556109 32013 log.cc:1196] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/81a5280811db4ad198b8a76ac5d7920b
I20260811 10:07:41.556350 32013 ts_tablet_manager.cc:1939] T 81a5280811db4ad198b8a76ac5d7920b P 9986f78a5bf0425db793f4a15b6ee79b: Deleting consensus metadata
I20260811 10:07:41.556636 31993 tablet_bootstrap.cc:492] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Bootstrap starting.
I20260811 10:07:41.556998 31993 tablet_bootstrap.cc:654] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.557303 31737 tablet_service.cc:1467] Processing CreateTablet for tablet f362f4da55cb41c3b98392458540fd74 (DEFAULT_TABLE table=test-table [id=5247f31846ee4a3c8a31aeb763395ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:07:41.557440 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f362f4da55cb41c3b98392458540fd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:41.557687 31993 tablet_bootstrap.cc:492] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: No bootstrap required, opened a new log
I20260811 10:07:41.557730 31993 ts_tablet_manager.cc:1397] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:41.557843 31993 raft_consensus.cc:348] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.557909 31993 raft_consensus.cc:374] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.557929 31993 raft_consensus.cc:729] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.557986 31993 consensus_queue.cc:260] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.558084 31993 ts_tablet_manager.cc:1428] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:41.558930 31990 tablet_bootstrap.cc:492] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Bootstrap starting.
I20260811 10:07:41.559159 31417 catalog_manager.cc:4841] TS 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905): tablet 81a5280811db4ad198b8a76ac5d7920b (table test-table [id=85c6dd7651ba48d680a2394b21af26b0]) successfully deleted
I20260811 10:07:41.559413 31990 tablet_bootstrap.cc:654] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:41.560248 31990 tablet_bootstrap.cc:492] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: No bootstrap required, opened a new log
I20260811 10:07:41.560298 31990 ts_tablet_manager.cc:1397] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:41.560467 31990 raft_consensus.cc:348] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.560539 31990 raft_consensus.cc:374] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:41.560555 31990 raft_consensus.cc:729] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f02ec8d04483b73b2fdc15ff8f9b, State: Initialized, Role: FOLLOWER
I20260811 10:07:41.560616 31990 consensus_queue.cc:260] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.560724 31990 ts_tablet_manager.cc:1428] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:41.676627 32040 raft_consensus.cc:482] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:07:41.676750 32040 raft_consensus.cc:504] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.676889 32040 leader_election.cc:288] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523), 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677)
I20260811 10:07:41.677184 31603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f362f4da55cb41c3b98392458540fd74" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" is_pre_election: true
I20260811 10:07:41.677299 31603 raft_consensus.cc:2454] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 0.
I20260811 10:07:41.677245 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f362f4da55cb41c3b98392458540fd74" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" is_pre_election: true
I20260811 10:07:41.677389 31758 raft_consensus.cc:2454] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 0.
I20260811 10:07:41.677477 31839 leader_election.cc:302] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cf266dba5d4e4abd29e32b8f501ff4, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:41.677626 32040 raft_consensus.cc:2781] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:07:41.677717 32040 raft_consensus.cc:482] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:07:41.677750 32040 raft_consensus.cc:3037] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.678529 32040 raft_consensus.cc:504] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.678692 32040 leader_election.cc:288] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 election: Requested vote from peers 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523), 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677)
I20260811 10:07:41.678990 31603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f362f4da55cb41c3b98392458540fd74" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cf266dba5d4e4abd29e32b8f501ff4"
I20260811 10:07:41.679011 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f362f4da55cb41c3b98392458540fd74" candidate_uuid: "9986f78a5bf0425db793f4a15b6ee79b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b"
I20260811 10:07:41.679095 31603 raft_consensus.cc:3037] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.679095 31758 raft_consensus.cc:3037] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:41.679966 31603 raft_consensus.cc:2454] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 1.
I20260811 10:07:41.679967 31758 raft_consensus.cc:2454] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9986f78a5bf0425db793f4a15b6ee79b in term 1.
I20260811 10:07:41.680219 31839 leader_election.cc:302] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51f02ec8d04483b73b2fdc15ff8f9b, 9986f78a5bf0425db793f4a15b6ee79b; no voters:
I20260811 10:07:41.680346 32040 raft_consensus.cc:2781] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:41.680420 32040 raft_consensus.cc:686] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 LEADER]: Becoming Leader. State: Replica: 9986f78a5bf0425db793f4a15b6ee79b, State: Running, Role: LEADER
I20260811 10:07:41.680476 32040 consensus_queue.cc:237] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } }
I20260811 10:07:41.681134 31433 catalog_manager.cc:5495] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3). New cstate: current_term: 1 leader_uuid: "9986f78a5bf0425db793f4a15b6ee79b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9986f78a5bf0425db793f4a15b6ee79b" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260811 10:07:41.709143 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.706720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42486 (local address 127.22.177.3:45905)
0811 10:07:41.706762 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.706764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.706775 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:07:41.706820 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.706820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.706829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.706870 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:41.706955 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.707487 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.708373 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.708567 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.708640 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.708843 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.708885 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.708942 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.709062 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.709080 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:07:41.714195 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.711840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45905 (local address 127.22.177.3:40217)
0811 10:07:41.711916 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.711929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:41.711981 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:41.712150 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:41.712151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:41.712153 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:07:41.712191 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.712192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.712847 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.712848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.713465 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:41.713468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.713873 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.713874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:41.713913 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.713918 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.714048 (+ 130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.714049 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:07:41.714084 (+ 35us) client_negotiation.cc:240] Negotiation successful
0811 10:07:41.714131 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":2}
I20260811 10:07:41.714350 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.711918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:40217 (local address 127.22.177.3:45905)
0811 10:07:41.711964 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:41.711965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:41.711974 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:41.712030 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:41.712031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:41.712039 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:41.712098 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:07:41.712273 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.712751 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.713568 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:41.713791 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:41.713852 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:41.713973 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:07:41.713996 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:07:41.714030 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:41.714087 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 10:07:41.714125 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":27}
I20260811 10:07:41.716435 31758 raft_consensus.cc:1261] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Refusing update from remote peer 9986f78a5bf0425db793f4a15b6ee79b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.716594 31603 raft_consensus.cc:1261] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Refusing update from remote peer 9986f78a5bf0425db793f4a15b6ee79b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:07:41.716673 31997 consensus_queue.cc:1035] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f02ec8d04483b73b2fdc15ff8f9b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.716777 32040 consensus_queue.cc:1035] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cf266dba5d4e4abd29e32b8f501ff4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:07:41.728101 31431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:41.728210 31431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:07:41.728931 31431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d10448ece2b8459b9a369e5714a6684b: Sending DeleteTablet for 3 replicas of tablet f362f4da55cb41c3b98392458540fd74
I20260811 10:07:41.729223 31888 tablet_service.cc:1514] Processing DeleteTablet for tablet f362f4da55cb41c3b98392458540fd74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:42458
I20260811 10:07:41.729234 31581 tablet_service.cc:1514] Processing DeleteTablet for tablet f362f4da55cb41c3b98392458540fd74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41920
I20260811 10:07:41.729322 32016 tablet_replica.cc:330] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: stopping tablet replica
I20260811 10:07:41.729322 32013 tablet_replica.cc:330] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: stopping tablet replica
I20260811 10:07:41.729301 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet f362f4da55cb41c3b98392458540fd74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:07:41 UTC) from {username='oryx', principal='oryx/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:58350
I20260811 10:07:41.729367 32013 raft_consensus.cc:2229] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:07:41.729367 32016 raft_consensus.cc:2229] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:41.729404 32016 raft_consensus.cc:2258] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.729409 32014 tablet_replica.cc:330] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: stopping tablet replica
I20260811 10:07:41.729452 32014 raft_consensus.cc:2229] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:07:41.729461 32013 raft_consensus.cc:2258] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.729488 32014 raft_consensus.cc:2258] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:07:41.729852 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 31512
I20260811 10:07:41.729946 32014 ts_tablet_manager.cc:1905] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.730024 32016 ts_tablet_manager.cc:1905] T f362f4da55cb41c3b98392458540fd74 P 17cf266dba5d4e4abd29e32b8f501ff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.731556 32014 ts_tablet_manager.cc:1918] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:41.731611 32014 log.cc:1196] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/f362f4da55cb41c3b98392458540fd74
I20260811 10:07:41.731853 32014 ts_tablet_manager.cc:1939] T f362f4da55cb41c3b98392458540fd74 P 4f51f02ec8d04483b73b2fdc15ff8f9b: Deleting consensus metadata
I20260811 10:07:41.732858 31420 catalog_manager.cc:4841] TS 4f51f02ec8d04483b73b2fdc15ff8f9b (127.22.177.2:39677): tablet f362f4da55cb41c3b98392458540fd74 (table test-table [id=5247f31846ee4a3c8a31aeb763395ad6]) successfully deleted
W20260811 10:07:41.740437 31420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:46523 (error 108)
I20260811 10:07:41.740607 32013 ts_tablet_manager.cc:1905] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:07:41.741410 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:41.741127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:46523 (local address 127.0.0.1:41930)
0811 10:07:41.741310 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:41.741360 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:46523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":42,"threads_started":1}
W20260811 10:07:41.741503 31420 catalog_manager.cc:4584] TS 17cf266dba5d4e4abd29e32b8f501ff4 (127.22.177.1:46523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f362f4da55cb41c3b98392458540fd74: Network error: Client connection negotiation failed: client connection to 127.22.177.1:46523: connect: Connection refused (error 111)
I20260811 10:07:41.741765 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 31666
I20260811 10:07:41.742681 32013 ts_tablet_manager.cc:1918] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:07:41.742774 32013 log.cc:1196] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/f362f4da55cb41c3b98392458540fd74
I20260811 10:07:41.743023 32013 ts_tablet_manager.cc:1939] T f362f4da55cb41c3b98392458540fd74 P 9986f78a5bf0425db793f4a15b6ee79b: Deleting consensus metadata
I20260811 10:07:41.749923 31417 catalog_manager.cc:4841] TS 9986f78a5bf0425db793f4a15b6ee79b (127.22.177.3:45905): tablet f362f4da55cb41c3b98392458540fd74 (table test-table [id=5247f31846ee4a3c8a31aeb763395ad6]) successfully deleted
I20260811 10:07:41.755383 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 31819
I20260811 10:07:41.764492 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 31399
2026-08-11T10:07:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2926 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[32084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[32084](info): set up 2 sockets
Aug 11 10:07:41 dist-test-slave-g350 krb5kdc[32084](info): commencing operation
krb5kdc: starting...
W20260811 10:07:42.978976 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:43 dist-test-slave-g350 krb5kdc[32084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:43.122138 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35831
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:35831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:43.206223 32097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:43.206496 32097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:43.206537 32097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:43.207719 32097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:43.207799 32097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:43.207825 32097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:43.207855 32097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:43.207885 32097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:43.209899 32097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:35831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:43.210270 32097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:43.210556 32097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:43.215828 32102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:43.215828 32105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:43.215907 32103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:43.216040 32097 server_base.cc:1034] running on GCE node
I20260811 10:07:43.216565 32097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:43.216887 32097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:43.218101 32097 hybrid_clock.cc:648] HybridClock initialized: now 1786442863218077 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T10:07:43Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1451 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:07:43 dist-test-slave-g350 krb5kdc[32111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:07:43 dist-test-slave-g350 krb5kdc[32111](info): set up 2 sockets
Aug 11 10:07:43 dist-test-slave-g350 krb5kdc[32111](info): commencing operation
krb5kdc: starting...
W20260811 10:07:44.337502 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:07:44 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:07:44.473188 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37063
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:44.560897 32124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:44.561149 32124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:44.561177 32124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:44.562593 32124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:44.562649 32124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:44.562665 32124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:44.562680 32124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:44.562696 32124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:44.564442 32124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:44.564754 32124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:44.565030 32124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:44.571041 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:44.571066 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:44.571041 32133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:44.571605 32124 server_base.cc:1034] running on GCE node
I20260811 10:07:44.571887 32124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:44.572199 32124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:44.573357 32124 hybrid_clock.cc:648] HybridClock initialized: now 1786442864573347 us; error 34 us; skew 500 ppm
Aug 11 10:07:44 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:44.576334 32124 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:44.577010 32124 webserver.cc:460] Webserver started at http://127.22.177.62:46297/ using document root <none> and password file <none>
I20260811 10:07:44.577366 32124 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:44.577451 32124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:44.577666 32124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:44.578869 32124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "ba8f082e33774d96a8e6bb890be5628a"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "21dff96ce89785a9229666820384fdfe"
server_key_iv: "1114e2efeefdc1b1485e380064b9a756"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.579320 32124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "ba8f082e33774d96a8e6bb890be5628a"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "21dff96ce89785a9229666820384fdfe"
server_key_iv: "1114e2efeefdc1b1485e380064b9a756"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.580978 32124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:44.582038 32140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:44.582301 32124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:44.582418 32124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ba8f082e33774d96a8e6bb890be5628a"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "21dff96ce89785a9229666820384fdfe"
server_key_iv: "1114e2efeefdc1b1485e380064b9a756"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.582504 32124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:44.601946 32124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:44.604274 32124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:44.604455 32124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:44.609082 32124 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37063
I20260811 10:07:44.609149 32196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37063 every 8 connection(s)
I20260811 10:07:44.609558 32124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:44.609894 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 32124
I20260811 10:07:44.610023 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:07:44.610299 23236 external_mini_cluster.cc:1428] Setting key 0bf5d346c2bdaf8308bc4ca829aed7d4
I20260811 10:07:44.610780 32197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:07:44.614308 32197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Bootstrap starting.
I20260811 10:07:44.614990 32197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Neither blocks nor log segments found. Creating new log.
I20260811 10:07:44.615315 32197 log.cc:824] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:44.616144 32197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: No bootstrap required, opened a new log
Aug 11 10:07:44 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:44.618256 32197 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:44.618438 32197 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:07:44.618505 32197 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8f082e33774d96a8e6bb890be5628a, State: Initialized, Role: FOLLOWER
I20260811 10:07:44.618680 32197 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:44.618778 32197 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:44.618814 32197 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:44.618870 32197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:07:44.619444 32197 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:44.619572 32197 leader_election.cc:302] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba8f082e33774d96a8e6bb890be5628a; no voters:
I20260811 10:07:44.619787 32197 leader_election.cc:288] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:07:44.619833 32202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:07:44.619948 32202 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 LEADER]: Becoming Leader. State: Replica: ba8f082e33774d96a8e6bb890be5628a, State: Running, Role: LEADER
I20260811 10:07:44.620088 32197 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:44.620105 32202 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:44.620502 32202 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba8f082e33774d96a8e6bb890be5628a. Latest consensus state: current_term: 1 leader_uuid: "ba8f082e33774d96a8e6bb890be5628a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } } }
I20260811 10:07:44.620519 32203 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba8f082e33774d96a8e6bb890be5628a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } } }
I20260811 10:07:44.620700 32203 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:44.620924 32202 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: This master's current role is: LEADER
W20260811 10:07:44.622227 32221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:07:44.622340 32221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:07:44.622431 32211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:44.622596 32211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:44.622632 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:44.610963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47556 (local address 127.22.177.62:37063)
0811 10:07:44.611173 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:44.611182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:44.611222 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:07:44.611961 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:44.611971 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:44.612036 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:44.612208 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:44.612875 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:44.614347 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:44.615260 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:44.615735 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:44.615896 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:44.618370 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:44.618379 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:44.618380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:44.618391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:44.620446 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:44.620905 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:44.620908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:44.620908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:44.621013 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:44.621260 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:44.621262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:44.621262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:44.621540 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:44.622317 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:44.622376 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 10:07:44.622482 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260811 10:07:44.624445 32211 catalog_manager.cc:1347] Generated new cluster ID: 8fad1ffb4711431f92b538127fddde3a
I20260811 10:07:44.624524 32211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:44.633147 32211 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:07:44.633813 32211 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:44.637691 32211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Generated new TSK 0
I20260811 10:07:44.637893 32211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:07:44.676929 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37063
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:44.762941 32227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:44.763177 32227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:44.763203 32227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:44.764498 32227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:44.764555 32227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:44.764583 32227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:44.766160 32227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:44.766436 32227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:44.766716 32227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:44.772255 32233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:44.772248 32232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:44.772354 32235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:44.772621 32227 server_base.cc:1034] running on GCE node
I20260811 10:07:44.772912 32227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:44.773190 32227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:44.774392 32227 hybrid_clock.cc:648] HybridClock initialized: now 1786442864774358 us; error 53 us; skew 500 ppm
Aug 11 10:07:44 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:44.777511 32227 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:07:44.778151 32227 webserver.cc:460] Webserver started at http://127.22.177.1:43401/ using document root <none> and password file <none>
I20260811 10:07:44.778436 32227 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:44.778512 32227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:44.778673 32227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:44.779788 32227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "c367f0dae57f46ac944ce69ab038772e"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "d1665e618839671239fde2d1a82d33f5"
server_key_iv: "65bd85d06a3e54671c688c0708399b73"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.780171 32227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "c367f0dae57f46ac944ce69ab038772e"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "d1665e618839671239fde2d1a82d33f5"
server_key_iv: "65bd85d06a3e54671c688c0708399b73"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.781716 32227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:44.782704 32242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:44.782920 32227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:44.783038 32227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "c367f0dae57f46ac944ce69ab038772e"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "d1665e618839671239fde2d1a82d33f5"
server_key_iv: "65bd85d06a3e54671c688c0708399b73"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.783120 32227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:44.793913 32227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:44.796429 32227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:44.796612 32227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:44.796876 32227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:44.797214 32227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:44.797274 32227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:44.797318 32227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:44.797346 32227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:44.803965 32227 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:34855
I20260811 10:07:44.804020 32364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:34855 every 8 connection(s)
I20260811 10:07:44.804409 32227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:07:44 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:44.813357 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 32227
I20260811 10:07:44.813493 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:07:44.813757 23236 external_mini_cluster.cc:1428] Setting key fb4c744ba2134d3813d7c8fb820719df
I20260811 10:07:44.815058 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:44.805257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:33243 (local address 127.22.177.62:37063)
0811 10:07:44.805318 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:44.805321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:44.805551 (+ 230us) server_negotiation.cc:407] Connection header received
0811 10:07:44.806118 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:44.806119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:44.806130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:44.806188 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:44.807147 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:44.807686 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:44.808696 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:44.808913 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:44.808973 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:44.811421 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:44.811423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:44.811423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:44.811431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:44.813035 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:44.813678 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:44.813679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:44.813680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:44.813754 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:44.814241 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:44.814243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:44.814243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:44.814397 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:44.814472 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:44.814895 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 10:07:44.814985 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:07:44.815318 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:44.805203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:33243)
0811 10:07:44.805409 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:44.805449 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:44.806009 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:44.806252 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:44.806256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:44.806850 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:44.807052 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:44.807056 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:44.807788 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:44.807789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:44.808608 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:44.808610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:44.808998 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:44.808999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:44.809104 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:44.809805 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:44.809814 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:44.811294 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:44.813134 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:44.813146 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:44.813149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:44.813521 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:44.813904 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:44.813905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:44.813905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:44.814053 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:44.814474 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:44.814481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:44.814694 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 10:07:44.814826 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 10:07:44.815030 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":135,"threads_started":1}
I20260811 10:07:44.815940 32365 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37063
I20260811 10:07:44.816066 32365 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:44.816316 32365 heartbeater.cc:511] Master 127.22.177.62:37063 requested a full tablet report, sending...
I20260811 10:07:44.816967 32157 ts_manager.cc:194] Registered new tserver with Master: c367f0dae57f46ac944ce69ab038772e (127.22.177.1:34855)
I20260811 10:07:44.817584 32157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:33243
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:07:44.846302 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37063
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:44.935441 32370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:44.935763 32370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:44.935796 32370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:44.937142 32370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:44.937194 32370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:44.937225 32370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:44.938879 32370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:44.939199 32370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:44.939502 32370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:44.945567 32379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:44.945562 32377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:44.945562 32376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:44.945869 32370 server_base.cc:1034] running on GCE node
I20260811 10:07:44.946157 32370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:44.946432 32370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:44.947634 32370 hybrid_clock.cc:648] HybridClock initialized: now 1786442864947628 us; error 40 us; skew 500 ppm
Aug 11 10:07:44 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:44.950610 32370 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:07:44.951049 32370 webserver.cc:460] Webserver started at http://127.22.177.2:37585/ using document root <none> and password file <none>
I20260811 10:07:44.951300 32370 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:44.951349 32370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:44.951500 32370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:44.952775 32370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "3599312d925f40c1889b54f1343e44c6"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "4751442b9882e26680818ec59ff68b71"
server_key_iv: "baa66bc7c9154fe839eed2934590c799"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.953163 32370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "3599312d925f40c1889b54f1343e44c6"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "4751442b9882e26680818ec59ff68b71"
server_key_iv: "baa66bc7c9154fe839eed2934590c799"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.954718 32370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:07:44.955709 32387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:44.956014 32370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:44.956144 32370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "3599312d925f40c1889b54f1343e44c6"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "4751442b9882e26680818ec59ff68b71"
server_key_iv: "baa66bc7c9154fe839eed2934590c799"
server_key_version: "encryptionkey@0"
I20260811 10:07:44.956233 32370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:44.991075 32370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:44.993953 32370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:44.994143 32370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:44.994428 32370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:44.994783 32370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:44.994843 32370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:44.994880 32370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:44.994906 32370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.001935 32370 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35835
I20260811 10:07:45.002007 32511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35835 every 8 connection(s)
I20260811 10:07:45.002477 32370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:07:45.003468 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 32370
I20260811 10:07:45.003597 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:07:45.003825 23236 external_mini_cluster.cc:1428] Setting key 6d7b6e01b2a8c84caaaba4efb5dca15b
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:07:45.014185 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.003536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:59307 (local address 127.22.177.62:37063)
0811 10:07:45.003596 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.003599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.003814 (+ 215us) server_negotiation.cc:407] Connection header received
0811 10:07:45.004468 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.004470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.004483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.004536 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.005798 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.006301 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.007355 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.007564 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.007652 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.010397 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:45.010399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:45.010400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:45.010410 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:45.012310 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:45.012823 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:45.012824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:45.012825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:45.012894 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:45.013270 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:45.013272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:45.013272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:45.013438 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:45.013518 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:45.013954 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 10:07:45.014064 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:07:45.014462 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.003355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:59307)
0811 10:07:45.003638 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.003684 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.004355 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.004625 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.004630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.005421 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.005657 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.005664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.006411 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.006412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.007248 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.007250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.007661 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.007663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.007796 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.008392 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.008401 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.010267 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:45.012452 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:45.012461 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:45.012463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:45.012733 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:45.012994 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:45.012995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:45.012996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:45.013130 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:45.013521 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:45.013528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:45.013771 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 10:07:45.013874 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 10:07:45.014024 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260811 10:07:45.015200 32512 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37063
I20260811 10:07:45.015348 32512 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:45.015539 32512 heartbeater.cc:511] Master 127.22.177.62:37063 requested a full tablet report, sending...
I20260811 10:07:45.015924 32157 ts_manager.cc:194] Registered new tserver with Master: 3599312d925f40c1889b54f1343e44c6 (127.22.177.2:35835)
I20260811 10:07:45.016417 32157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:59307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:07:45.037925 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37063
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:45.129293 32517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:45.129565 32517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:45.129671 32517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:45.131198 32517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:45.131274 32517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:45.131307 32517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:45.133067 32517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:45.133366 32517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:45.133698 32517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:45.139653 32523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.139807 32517 server_base.cc:1034] running on GCE node
W20260811 10:07:45.139683 32524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:45.139658 32526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.140240 32517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:45.140508 32517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:45.141671 32517 hybrid_clock.cc:648] HybridClock initialized: now 1786442865141648 us; error 37 us; skew 500 ppm
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:45.144572 32517 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:07:45.145047 32517 webserver.cc:460] Webserver started at http://127.22.177.3:37963/ using document root <none> and password file <none>
I20260811 10:07:45.145318 32517 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:45.145370 32517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:45.145509 32517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:07:45.146675 32517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "fba3582edb734b088dee85b586589704"
format_stamp: "Formatted at 2026-08-11 10:07:45 on dist-test-slave-g350"
server_key: "a38d787b7e80460ed73039f04d1a8bcb"
server_key_iv: "2e1df44d5f01f0ffd25bc1225be04773"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.147047 32517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "fba3582edb734b088dee85b586589704"
format_stamp: "Formatted at 2026-08-11 10:07:45 on dist-test-slave-g350"
server_key: "a38d787b7e80460ed73039f04d1a8bcb"
server_key_iv: "2e1df44d5f01f0ffd25bc1225be04773"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.148532 32517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:07:45.149500 32534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.149807 32517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:45.149966 32517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "fba3582edb734b088dee85b586589704"
format_stamp: "Formatted at 2026-08-11 10:07:45 on dist-test-slave-g350"
server_key: "a38d787b7e80460ed73039f04d1a8bcb"
server_key_iv: "2e1df44d5f01f0ffd25bc1225be04773"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.150070 32517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:45.166138 32517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:45.168905 32517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:45.169108 32517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:45.169413 32517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:45.169862 32517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:45.169926 32517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.169984 32517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:45.170019 32517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.177050 32517 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:42735
I20260811 10:07:45.177094 32657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:42735 every 8 connection(s)
I20260811 10:07:45.177465 32517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:07:45.184759 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 32517
I20260811 10:07:45.184859 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:07:45.185103 23236 external_mini_cluster.cc:1428] Setting key 89a7525154aa6c24fd1a13da6730a1e1
I20260811 10:07:45.188558 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.178439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:39179 (local address 127.22.177.62:37063)
0811 10:07:45.178542 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.178545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.178719 (+ 174us) server_negotiation.cc:407] Connection header received
0811 10:07:45.179293 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.179295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.179307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.179370 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.180443 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.180980 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.182229 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.182428 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.182500 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.184974 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:45.184976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:45.184977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:45.184987 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:45.187032 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:45.187553 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:45.187554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:45.187555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:45.187627 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:45.187918 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:45.187919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:45.187919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:45.188041 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:45.188099 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:45.188413 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 10:07:45.188489 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 10:07:45.188823 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.178385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:39179)
0811 10:07:45.178579 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.178617 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.179142 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.179481 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.179485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.180096 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.180317 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.180322 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.181122 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.181123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.182063 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.182066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.182538 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.182539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.182655 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.183258 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.183267 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.184860 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:07:45.187119 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:45.187127 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:45.187129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:45.187456 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:45.187721 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:07:45.187722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:07:45.187723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:07:45.187850 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:07:45.188087 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:07:45.188091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:07:45.188288 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 10:07:45.188383 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 10:07:45.188525 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260811 10:07:45.189496 32658 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37063
I20260811 10:07:45.189702 32658 heartbeater.cc:464] Registering TS with master...
I20260811 10:07:45.189950 32658 heartbeater.cc:511] Master 127.22.177.62:37063 requested a full tablet report, sending...
I20260811 10:07:45.190464 32157 ts_manager.cc:194] Registered new tserver with Master: fba3582edb734b088dee85b586589704 (127.22.177.3:42735)
I20260811 10:07:45.190861 32157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:39179
I20260811 10:07:45.197744 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab.
Entry for principal oryx/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab.
WARNING: no policy specified for oryx/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab.
Entry for principal oryx/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab.
WARNING: no policy specified for oryx/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab.
Entry for principal oryx/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab.
I20260811 10:07:45.292544 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 32227
I20260811 10:07:45.300365 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 32370
I20260811 10:07:45.308553 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 32517
I20260811 10:07:45.316344 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 32124
I20260811 10:07:45.324884 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37063
--webserver_interface=127.22.177.62
--webserver_port=46297
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:07:45.411375 32668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:45.411593 32668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:45.411618 32668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:45.412819 32668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:07:45.412868 32668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:45.412882 32668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:45.412895 32668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:07:45.412909 32668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:07:45.414506 32668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=46297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:45.414775 32668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:45.415037 32668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:45.420416 32674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:45.420428 32675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:45.420480 32677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.420789 32668 server_base.cc:1034] running on GCE node
I20260811 10:07:45.421096 32668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:45.421373 32668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:45.422600 32668 hybrid_clock.cc:648] HybridClock initialized: now 1786442865422562 us; error 51 us; skew 500 ppm
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:45.428678 32668 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:07:45.429221 32668 webserver.cc:460] Webserver started at http://127.22.177.62:46297/ using document root <none> and password file <none>
I20260811 10:07:45.429520 32668 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:45.429656 32668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:45.431185 32668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:45.431979 32685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.432206 32668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:45.432337 32668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ba8f082e33774d96a8e6bb890be5628a"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "21dff96ce89785a9229666820384fdfe"
server_key_iv: "1114e2efeefdc1b1485e380064b9a756"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.432731 32668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:45.447083 32668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:45.449873 32668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:45.450098 32668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:45.455037 32668 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37063
I20260811 10:07:45.455097 32739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37063 every 8 connection(s)
I20260811 10:07:45.455528 32668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:07:45.457166 32740 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:07:45.457934 32740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Bootstrap starting.
I20260811 10:07:45.460906 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 32668
I20260811 10:07:45.461256 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:34855
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:37063
--webserver_port=43401
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:07:45.467464 32740 log.cc:824] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:07:45.469272 32740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:07:45.469663 32740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Bootstrap complete.
I20260811 10:07:45.472047 32740 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:45.472281 32740 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8f082e33774d96a8e6bb890be5628a, State: Initialized, Role: FOLLOWER
I20260811 10:07:45.472462 32740 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:45.472568 32740 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:07:45.472630 32740 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:07:45.472679 32740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:07:45.473812 32740 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:45.473941 32740 leader_election.cc:302] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba8f082e33774d96a8e6bb890be5628a; no voters:
I20260811 10:07:45.474169 32740 leader_election.cc:288] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:07:45.474206 32745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:07:45.474339 32745 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [term 2 LEADER]: Becoming Leader. State: Replica: ba8f082e33774d96a8e6bb890be5628a, State: Running, Role: LEADER
I20260811 10:07:45.474465 32740 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:07:45.474499 32745 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } }
I20260811 10:07:45.474908 32747 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba8f082e33774d96a8e6bb890be5628a. Latest consensus state: current_term: 2 leader_uuid: "ba8f082e33774d96a8e6bb890be5628a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } } }
I20260811 10:07:45.475001 32747 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:45.474949 32746 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ba8f082e33774d96a8e6bb890be5628a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8f082e33774d96a8e6bb890be5628a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37063 } } }
I20260811 10:07:45.475066 32746 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a [sys.catalog]: This master's current role is: LEADER
I20260811 10:07:45.475232 32752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:07:45.475886 32752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:07:45.476101 32752 catalog_manager.cc:1259] Loaded cluster ID: 8fad1ffb4711431f92b538127fddde3a
I20260811 10:07:45.476363 32752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:07:45.477702 32752 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:07:45.477907 32752 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba8f082e33774d96a8e6bb890be5628a: Loaded TSK: 0
I20260811 10:07:45.478125 32752 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:07:45.551864 32742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:45.552178 32742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:45.552279 32742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:45.553570 32742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:45.553695 32742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:45.553730 32742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:07:45.556555 32742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.1.keytab
--principal=oryx/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:34855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=43401
--tserver_master_addrs=127.22.177.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:45.556972 32742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:45.557341 32742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:45.562904 303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:45.562945 302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.563133 32742 server_base.cc:1034] running on GCE node
W20260811 10:07:45.563218 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.563444 32742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:45.563750 32742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:45.564960 32742 hybrid_clock.cc:648] HybridClock initialized: now 1786442865564944 us; error 28 us; skew 500 ppm
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442865, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:45.568177 32742 init.cc:377] Logged in from keytab as oryx/127.22.177.1@KRBTEST.COM (short username oryx)
I20260811 10:07:45.568745 32742 webserver.cc:460] Webserver started at http://127.22.177.1:43401/ using document root <none> and password file <none>
I20260811 10:07:45.569005 32742 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:45.569096 32742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:45.570688 32742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:45.571385 313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.571593 32742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:45.571722 32742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "c367f0dae57f46ac944ce69ab038772e"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "d1665e618839671239fde2d1a82d33f5"
server_key_iv: "65bd85d06a3e54671c688c0708399b73"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.572091 32742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:45.590873 32742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:45.593403 32742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:45.593528 32742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:45.593839 32742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:45.594190 32742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:45.594257 32742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.594295 32742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:45.594348 32742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.601042 32742 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:34855
I20260811 10:07:45.601087 436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:34855 every 8 connection(s)
I20260811 10:07:45.601452 32742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:07:45.608482 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 32742
I20260811 10:07:45.608815 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:35835
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:37063
--webserver_port=37585
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.611847 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.602251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:59865)
0811 10:07:45.602466 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.602514 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.603141 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.603616 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.603622 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.604441 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.604760 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.604768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.605961 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.605962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.607230 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.607233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.607807 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.607808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.607955 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.608755 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.608763 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.611606 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":120,"threads_started":1}
W20260811 10:07:45.612000 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.602407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:59865 (local address 127.22.177.62:37063)
0811 10:07:45.602573 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.602581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.602622 (+ 41us) server_negotiation.cc:407] Connection header received
0811 10:07:45.603377 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.603382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.603425 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.603533 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.604912 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.605857 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.607328 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.607713 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.607797 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.611767 (+ 3970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:59865: BlockingRecv error: recv got EOF from 127.22.177.1:59865 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
W20260811 10:07:45.612387 437 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:37063 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.177.62:37063: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.617453 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.612685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:33577)
0811 10:07:45.612733 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.612743 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.612785 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.612968 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.612968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.613111 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.613166 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.613168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.613888 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.613889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.614505 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.614508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.614866 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.614867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.614936 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.615411 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.615417 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.617360 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 10:07:45.617522 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.612732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:33577 (local address 127.22.177.62:37063)
0811 10:07:45.612778 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.612780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.612790 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:45.612849 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.612850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.612863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.612964 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.613247 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.613798 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.614592 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.614789 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.614869 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.617443 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:33577: BlockingRecv error: recv got EOF from 127.22.177.1:33577 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.622553 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.617768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:44189)
0811 10:07:45.617948 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.617959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.618043 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.618176 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.618177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.618362 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.618421 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.618423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.619081 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.619082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.619679 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.619681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.620075 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.620076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.620142 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.620642 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.620647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.622461 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":126,"threads_started":1}
W20260811 10:07:45.622706 437 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:07:45.622725 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.617814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:44189 (local address 127.22.177.62:37063)
0811 10:07:45.617854 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.617856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.618026 (+ 170us) server_negotiation.cc:407] Connection header received
0811 10:07:45.618104 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.618105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.618117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.618173 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.618514 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.618994 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.619771 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.619965 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.620043 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.622630 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:44189: BlockingRecv error: recv got EOF from 127.22.177.1:44189 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 10:07:45.696745 441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:45.696961 441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:45.696986 441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:45.698194 441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:45.698238 441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:45.698264 441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:07:45.699694 441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.2.keytab
--principal=oryx/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:35835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=37585
--tserver_master_addrs=127.22.177.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:45.699901 441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:45.700088 441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:45.705736 450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:45.705730 449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.705950 441 server_base.cc:1034] running on GCE node
W20260811 10:07:45.705730 452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.706277 441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:45.706542 441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:45.707751 441 hybrid_clock.cc:648] HybridClock initialized: now 1786442865707718 us; error 48 us; skew 500 ppm
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442865, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:45.710665 441 init.cc:377] Logged in from keytab as oryx/127.22.177.2@KRBTEST.COM (short username oryx)
I20260811 10:07:45.711161 441 webserver.cc:460] Webserver started at http://127.22.177.2:37585/ using document root <none> and password file <none>
I20260811 10:07:45.711431 441 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:45.711529 441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:45.713007 441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:07:45.713708 459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.713900 441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:07:45.713999 441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "3599312d925f40c1889b54f1343e44c6"
format_stamp: "Formatted at 2026-08-11 10:07:44 on dist-test-slave-g350"
server_key: "4751442b9882e26680818ec59ff68b71"
server_key_iv: "baa66bc7c9154fe839eed2934590c799"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.714372 441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:45.736761 441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:45.739128 441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:45.739341 441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:45.739624 441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:45.739967 441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:45.740025 441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.740068 441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:45.740096 441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.746579 441 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35835
I20260811 10:07:45.746661 581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35835 every 8 connection(s)
I20260811 10:07:45.747056 441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.755076 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.747885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:37009)
0811 10:07:45.748090 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.748134 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.748623 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.748856 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.748859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.749452 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.749682 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.749687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.750427 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.750428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.751258 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.751261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.751635 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.751636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.751747 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.752328 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.752337 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.754908 (+ 2571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
W20260811 10:07:45.755110 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.747972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:37009 (local address 127.22.177.62:37063)
0811 10:07:45.748059 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.748061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.748231 (+ 170us) server_negotiation.cc:407] Connection header received
0811 10:07:45.748712 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.748713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.748727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.748802 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.749793 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.750323 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.751364 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.751549 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.751610 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.755046 (+ 3436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:37009: BlockingRecv error: recv got EOF from 127.22.177.2:37009 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 10:07:45.755569 582 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:37063 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.177.62:37063: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
I20260811 10:07:45.756851 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 441
I20260811 10:07:45.757202 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:42735
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:37063
--webserver_port=37963
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.761900 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.755903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:60411)
0811 10:07:45.755997 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.756012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.756081 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.756246 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.756247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.756394 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.756474 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.756476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.757530 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.757531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.758417 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.758420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.758866 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.758867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.758945 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.759539 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.759544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.761799 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 10:07:45.761972 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.756012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:60411 (local address 127.22.177.62:37063)
0811 10:07:45.756081 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.756083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.756093 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:45.756137 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.756137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.756146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.756210 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.756616 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.757443 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.758535 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.758769 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.758844 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.761899 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:60411: BlockingRecv error: recv got EOF from 127.22.177.2:60411 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.767655 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.762168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:54405)
0811 10:07:45.762370 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.762382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.762429 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.762657 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.762658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.762823 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.762866 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.762868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.763568 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.763569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.764452 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.764454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.764845 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.764846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.764913 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.765384 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.765390 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.767553 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":112,"threads_started":1}
W20260811 10:07:45.767722 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.762459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:54405 (local address 127.22.177.62:37063)
0811 10:07:45.762515 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.762517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.762532 (+ 15us) server_negotiation.cc:407] Connection header received
0811 10:07:45.762557 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.762558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.762566 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.762621 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.762952 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.763478 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.764556 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.764759 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.764825 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.767650 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:54405: BlockingRecv error: recv got EOF from 127.22.177.2:54405 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260811 10:07:45.767848 582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:07:45.845242 586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:07:45.845463 586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:07:45.845489 586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:07:45.846735 586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:07:45.846784 586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:07:45.846812 586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:07:45.848253 586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/krb5kdc/oryx_127.22.177.3.keytab
--principal=oryx/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:42735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=37963
--tserver_master_addrs=127.22.177.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:07:45.848471 586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:07:45.848658 586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:07:45.854920 592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.855115 586 server_base.cc:1034] running on GCE node
W20260811 10:07:45.854943 593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:07:45.854950 595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:07:45.855470 586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:07:45.855742 586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:07:45.856936 586 hybrid_clock.cc:648] HybridClock initialized: now 1786442865856921 us; error 53 us; skew 500 ppm
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442865, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:07:45.860203 586 init.cc:377] Logged in from keytab as oryx/127.22.177.3@KRBTEST.COM (short username oryx)
I20260811 10:07:45.860730 586 webserver.cc:460] Webserver started at http://127.22.177.3:37963/ using document root <none> and password file <none>
I20260811 10:07:45.861029 586 fs_manager.cc:359] Metadata directory not provided
I20260811 10:07:45.861129 586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:07:45.863078 586 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:45.863884 602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.864118 586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:07:45.864238 586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "fba3582edb734b088dee85b586589704"
format_stamp: "Formatted at 2026-08-11 10:07:45 on dist-test-slave-g350"
server_key: "a38d787b7e80460ed73039f04d1a8bcb"
server_key_iv: "2e1df44d5f01f0ffd25bc1225be04773"
server_key_version: "encryptionkey@0"
I20260811 10:07:45.864682 586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:07:45.901713 586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:07:45.904268 586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:07:45.904466 586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:07:45.904735 586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:07:45.905074 586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:07:45.905133 586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.905170 586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:07:45.905210 586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:07:45.911645 586 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:42735
I20260811 10:07:45.911695 728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:42735 every 8 connection(s)
I20260811 10:07:45.912060 586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:07:45.915495 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 586
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:07:45.920770 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.912829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52887 (local address 127.22.177.62:37063)
0811 10:07:45.912911 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.912914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.913184 (+ 270us) server_negotiation.cc:407] Connection header received
0811 10:07:45.913780 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.913782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.913796 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.913851 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.915009 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.915497 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.917077 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.917365 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.917431 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.920693 (+ 3262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:52887: BlockingRecv error: recv got EOF from 127.22.177.3:52887 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:07:45.920807 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.912844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:52887)
0811 10:07:45.913046 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.913086 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.913588 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.913906 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.913910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.914536 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.914751 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.914756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.915615 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.915616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.916990 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.916994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.917460 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.917461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.917603 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.918188 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.918196 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.920586 (+ 2390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
W20260811 10:07:45.921046 730 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:37063 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.177.62:37063: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
I20260811 10:07:45.923892 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.916097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.22.177.62:37063)
0811 10:07:45.916216 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.916218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.916227 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:45.916328 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.916328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.916338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.916394 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.917152 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.917830 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.918632 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.918858 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.918936 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.920223 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:07:45.920227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:07:45.920228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:07:45.920239 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:07:45.922408 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:45.922852 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:45.922856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:45.922856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:45.922949 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:07:45.923163 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:07:45.923164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:07:45.923164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:07:45.923386 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:07:45.923535 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 10:07:45.923642 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 10:07:45.923749 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.926329 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.921316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:39415)
0811 10:07:45.921371 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.921383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.921433 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.921576 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.921576 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.921764 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.921816 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.921818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.922682 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.922683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.923439 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.923442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.923876 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.923877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.923927 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.924456 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.924462 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.926277 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 10:07:45.926474 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.921352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:39415 (local address 127.22.177.62:37063)
0811 10:07:45.921388 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.921391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.921419 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:07:45.921492 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.921493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.921503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.921573 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.921898 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.922600 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.923527 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.923796 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.923866 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.926382 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:39415: BlockingRecv error: recv got EOF from 127.22.177.3:39415 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:45 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:45.930732 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:45.926633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:49531)
0811 10:07:45.926667 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:45.926678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:45.926725 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:45.926826 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:45.926827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:45.926930 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:45.926957 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.926957 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.927637 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.927638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.928107 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:45.928108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.928472 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.928473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:45.928521 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.928878 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:45.928882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:45.930631 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260811 10:07:45.930769 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:45.926688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:49531 (local address 127.22.177.62:37063)
0811 10:07:45.926742 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:45.926744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:45.926752 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:07:45.926769 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:45.926770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:45.926777 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:45.926822 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:45.927093 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.927554 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.928189 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:45.928375 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:45.928431 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:45.930710 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:49531: BlockingRecv error: recv got EOF from 127.22.177.3:49531 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 10:07:45.930918 730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 10:07:46 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:46 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:46.628906 735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:46.623186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:49767)
0811 10:07:46.623357 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:46.623374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:46.623440 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:46.623737 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:46.623738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:46.623915 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:46.623977 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:46.623979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.624821 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.624822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:46.625382 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:46.625384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.625895 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.625897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:46.625950 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:46.626537 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:46.626541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:46.628843 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
W20260811 10:07:46.629101 736 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:46.623403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:49767 (local address 127.22.177.62:37063)
0811 10:07:46.623576 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:46.623578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:46.623588 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:46.623613 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:46.623613 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:46.623623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:46.623695 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:46.624088 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.624689 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.625490 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.625797 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.625864 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:46.629021 (+ 3157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:49767: BlockingRecv error: recv got EOF from 127.22.177.1:49767 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":119,"threads_started":1}
Aug 11 10:07:46 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:46 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:46.774281 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:46.768293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:46809)
0811 10:07:46.768434 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:46.768445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:46.768496 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:46.768743 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:46.768745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:46.768945 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:46.769004 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:46.769006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.770043 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.770044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:46.770755 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:46.770758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.771157 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.771157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:46.771207 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:46.771831 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:46.771836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:46.774109 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260811 10:07:46.774315 736 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:46.768410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:46809 (local address 127.22.177.62:37063)
0811 10:07:46.768474 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:46.768480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:46.768490 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:46.768556 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:46.768557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:46.768568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:46.768627 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:46.769210 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.769935 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.770881 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.771078 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.771139 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:46.774257 (+ 3118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:46809: BlockingRecv error: recv got EOF from 127.22.177.2:46809 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 10:07:46 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:46 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:46.937376 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:46.931544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:32953)
0811 10:07:46.931697 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:46.931711 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:46.931775 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:46.931977 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:46.931978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:46.932153 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:46.932204 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:46.932205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.933323 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.933324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:46.934008 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:46.934011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.934481 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.934482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:46.934549 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:46.935167 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:46.935173 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:46.937300 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
W20260811 10:07:46.937542 736 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:46.931641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:32953 (local address 127.22.177.62:37063)
0811 10:07:46.931764 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:46.931766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:46.931780 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:07:46.931851 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:46.931852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:46.931866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:46.931915 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:46.932446 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.933216 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.934166 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:46.934390 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:46.934481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:46.937455 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:32953: BlockingRecv error: recv got EOF from 127.22.177.3:32953 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 10:07:47 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:47 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:47.635142 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:47.629586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:42931)
0811 10:07:47.629768 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:47.629789 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:47.629868 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:47.630076 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:47.630077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:47.630365 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:47.630417 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:47.630419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.631131 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.631132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:47.631766 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:47.631769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.632205 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.632206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:47.632327 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:47.632854 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:47.632859 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:47.635079 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260811 10:07:47.635339 740 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:47.629766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:42931 (local address 127.22.177.62:37063)
0811 10:07:47.629914 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:47.629916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:47.629925 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:47.629950 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:47.629950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:47.629960 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:47.630024 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:47.630505 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.631029 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.631882 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.632084 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.632144 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:47.635221 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:42931: BlockingRecv error: recv got EOF from 127.22.177.1:42931 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
Aug 11 10:07:47 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:47 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:47.780428 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:47.774961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:39005)
0811 10:07:47.775107 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:47.775120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:47.775193 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:47.775481 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:47.775482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:47.775686 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:47.775741 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:47.775742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.776629 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.776630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:47.777187 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:47.777189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.777674 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.777675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:47.777741 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:47.778298 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:47.778303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:47.780324 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
W20260811 10:07:47.780486 740 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:47.775030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:39005 (local address 127.22.177.62:37063)
0811 10:07:47.775097 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:47.775099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:47.775260 (+ 161us) server_negotiation.cc:407] Connection header received
0811 10:07:47.775345 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:47.775346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:47.775362 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:47.775418 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:47.775900 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.776524 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.777289 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.777484 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.777549 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:47.780419 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:39005: BlockingRecv error: recv got EOF from 127.22.177.2:39005 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 10:07:47 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:47 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:47.945140 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:47.937953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:50739)
0811 10:07:47.938100 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:47.938195 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:47.938290 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:47.938462 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:47.938464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:47.938655 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:47.938734 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:47.938736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.939989 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.939990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:47.941009 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:47.941012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.941473 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.941475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:47.941570 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:47.942331 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:47.942338 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:47.945038 (+ 2700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
W20260811 10:07:47.945227 740 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:47.938095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:50739 (local address 127.22.177.62:37063)
0811 10:07:47.938168 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:47.938170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:47.938268 (+ 98us) server_negotiation.cc:407] Connection header received
0811 10:07:47.938359 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:47.938360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:47.938377 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:47.938442 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:47.938845 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.939876 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.941138 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:47.941376 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:47.941440 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:47.945148 (+ 3708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:50739: BlockingRecv error: recv got EOF from 127.22.177.3:50739 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 10:07:48 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:48 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:48.640865 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:48.635836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:44061)
0811 10:07:48.636006 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:48.636016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:48.636089 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:48.636425 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:48.636426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:48.636603 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:48.636655 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:48.636656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.637402 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.637403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:48.637961 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:48.637963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.638346 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.638346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:48.638393 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:48.638865 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:48.638870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:48.640780 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
W20260811 10:07:48.641053 745 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:48.635858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:44061 (local address 127.22.177.62:37063)
0811 10:07:48.636015 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:48.636017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:48.636093 (+ 76us) server_negotiation.cc:407] Connection header received
0811 10:07:48.636273 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:48.636274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:48.636292 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:48.636374 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:48.636751 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.637282 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.638045 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.638264 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.638329 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:48.640974 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:44061: BlockingRecv error: recv got EOF from 127.22.177.1:44061 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
Aug 11 10:07:48 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:48 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:48.786226 746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:48.781047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:52113)
0811 10:07:48.781205 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:48.781219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:48.781296 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:48.781496 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:48.781498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:48.781799 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:48.781852 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:48.781854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.782595 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.782596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:48.783205 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:48.783207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.783579 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.783580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:48.783637 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:48.784134 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:48.784140 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:48.786153 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":143,"threads_started":1}
W20260811 10:07:48.786423 745 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:48.781118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:52113 (local address 127.22.177.62:37063)
0811 10:07:48.781219 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:48.781222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:48.781301 (+ 79us) server_negotiation.cc:407] Connection header received
0811 10:07:48.781364 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:48.781365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:48.781379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:48.781428 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:48.781961 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.782483 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.783294 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.783484 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.783544 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:48.786349 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:52113: BlockingRecv error: recv got EOF from 127.22.177.2:52113 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 10:07:48 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:48 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:48.951867 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:48.945966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:55341)
0811 10:07:48.946138 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:48.946154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:48.946223 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:48.946450 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:48.946451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:48.946634 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:48.946692 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:48.946694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.947705 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.947706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:48.948291 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:48.948294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.948711 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.948713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:48.948769 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:48.949410 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:48.949418 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:48.951786 (+ 2368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
W20260811 10:07:48.951946 745 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:48.946148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:55341 (local address 127.22.177.62:37063)
0811 10:07:48.946230 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:48.946232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:48.946244 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:48.946279 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:48.946280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:48.946294 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:48.946423 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:48.946808 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.947606 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.948400 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:48.948598 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:48.948660 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:48.951894 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:55341: BlockingRecv error: recv got EOF from 127.22.177.3:55341 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 10:07:49 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:49 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:49.647353 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:49.641466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:42159)
0811 10:07:49.641707 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:49.641731 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:49.641826 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:49.642238 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:49.642239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:49.642417 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:49.642469 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:49.642471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.643161 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.643163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:49.643762 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:49.643764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.644283 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.644285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:49.644364 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:49.645044 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:49.645049 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:49.647291 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":80,"threads_started":1}
W20260811 10:07:49.647583 749 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:49.641665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:42159 (local address 127.22.177.62:37063)
0811 10:07:49.641999 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:49.642001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:49.642054 (+ 53us) server_negotiation.cc:407] Connection header received
0811 10:07:49.642091 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:49.642092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:49.642108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:49.642183 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:49.642551 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.643059 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.643864 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.644167 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.644243 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:49.647509 (+ 3266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:42159: BlockingRecv error: recv got EOF from 127.22.177.1:42159 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 11 10:07:49 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:49 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:49.792383 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:49.786895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:53889)
0811 10:07:49.787065 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:49.787078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:49.787136 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:49.787338 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:49.787339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:49.787513 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:49.787565 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:49.787567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.788332 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.788333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:49.788875 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:49.788877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.789313 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.789315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:49.789368 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:49.790003 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:49.790009 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:49.792321 (+ 2312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
W20260811 10:07:49.792546 749 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:49.787005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:53889 (local address 127.22.177.62:37063)
0811 10:07:49.787131 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:49.787134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:49.787154 (+ 20us) server_negotiation.cc:407] Connection header received
0811 10:07:49.787189 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:49.787190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:49.787202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:49.787279 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:49.787642 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.788191 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.788993 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.789205 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.789273 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:49.792450 (+ 3177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:53889: BlockingRecv error: recv got EOF from 127.22.177.2:53889 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 10:07:49 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:49 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:49.958669 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:49.952861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:48911)
0811 10:07:49.953080 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:49.953094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:49.953170 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:49.953386 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:49.953388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:49.953743 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:49.953816 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:49.953818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.954592 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.954594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:49.955178 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:49.955179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.955563 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.955564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:49.955624 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:49.956166 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:49.956172 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:49.958543 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
W20260811 10:07:49.958802 749 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:49.953038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:48911 (local address 127.22.177.62:37063)
0811 10:07:49.953146 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:49.953148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:49.953160 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:49.953261 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:49.953262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:49.953282 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:49.953377 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:49.953912 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.954494 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.955273 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:49.955482 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:49.955564 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:49.958729 (+ 3165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:48911: BlockingRecv error: recv got EOF from 127.22.177.3:48911 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 10:07:50 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:50 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:50.657446 752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:50.647922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:48049)
0811 10:07:50.648087 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:50.648098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:50.648155 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:50.648289 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:50.648290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:50.648510 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:50.648561 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:50.648563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.649250 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.649250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:50.649833 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:50.649835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.650226 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.650227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:50.650272 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:50.654741 (+ 4469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:50.654749 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:50.657343 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
W20260811 10:07:50.657544 753 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:50.648016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:48049 (local address 127.22.177.62:37063)
0811 10:07:50.648152 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:50.648154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:50.648172 (+ 18us) server_negotiation.cc:407] Connection header received
0811 10:07:50.648196 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:50.648197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:50.648207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:50.648274 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:50.648653 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.649178 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.649937 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.650140 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.650213 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:50.657464 (+ 7251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:48049: BlockingRecv error: recv got EOF from 127.22.177.1:48049 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
Aug 11 10:07:50 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:50 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:50.798132 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:50.792921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:47787)
0811 10:07:50.793157 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:50.793189 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:50.793268 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:50.793510 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:50.793511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:50.793773 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:50.793827 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:50.793830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.794518 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.794519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:50.795067 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:50.795069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.795459 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.795461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:50.795514 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:50.796020 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:50.796025 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:50.798069 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":78,"threads_started":1}
W20260811 10:07:50.798281 753 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:50.793071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:47787 (local address 127.22.177.62:37063)
0811 10:07:50.793142 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:50.793144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:50.793274 (+ 130us) server_negotiation.cc:407] Connection header received
0811 10:07:50.793389 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:50.793390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:50.793401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:50.793453 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:50.793910 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.794424 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.795146 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.795335 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.795390 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:50.798198 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:47787: BlockingRecv error: recv got EOF from 127.22.177.2:47787 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 11 10:07:50 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:50 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:50.964834 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:50.959441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:36343)
0811 10:07:50.959591 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:50.959607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:50.959669 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:50.959906 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:50.959908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:50.960109 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:50.960164 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:50.960165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.960977 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.960978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:50.961684 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:50.961687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.962124 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.962126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:50.962254 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:50.962851 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:50.962857 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:50.964733 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260811 10:07:50.964895 753 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:50.959569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:36343 (local address 127.22.177.62:37063)
0811 10:07:50.959704 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:50.959706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:50.959718 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:50.959748 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:50.959749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:50.959763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:50.959837 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:50.960301 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.960866 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.961772 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:50.962044 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:50.962115 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:50.964831 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:36343: BlockingRecv error: recv got EOF from 127.22.177.3:36343 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 11 10:07:51 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:51 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:51.663841 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:51.658183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:58661)
0811 10:07:51.658567 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:51.658579 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:51.658692 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:51.658921 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:51.658923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:51.659108 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:51.659161 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:51.659163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.660009 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.660010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:51.660610 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:51.660613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.661018 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.661019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:51.661081 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:51.661657 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:51.661663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:51.663733 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":225,"threads_started":1}
W20260811 10:07:51.663930 759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:51.658417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:58661 (local address 127.22.177.62:37063)
0811 10:07:51.658547 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:51.658550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:51.658737 (+ 187us) server_negotiation.cc:407] Connection header received
0811 10:07:51.658779 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:51.658780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:51.658795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:51.658860 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:51.659269 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.659905 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.660714 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.660922 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.661014 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:51.663851 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:58661: BlockingRecv error: recv got EOF from 127.22.177.1:58661 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
Aug 11 10:07:51 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:51 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:51.803757 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:51.798776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:43287)
0811 10:07:51.798922 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:51.798936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:51.799005 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:51.799164 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:51.799165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:51.799367 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:51.799433 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:51.799435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.800138 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.800140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:51.800690 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:51.800692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.801094 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.801095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:51.801175 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:51.801740 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:51.801755 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:51.803649 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
W20260811 10:07:51.803818 759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:51.798908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:43287 (local address 127.22.177.62:37063)
0811 10:07:51.798967 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:51.798969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:51.798983 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:07:51.799075 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:51.799076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:51.799093 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:51.799154 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:51.799527 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.800043 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.800812 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.801014 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.801092 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:51.803752 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:43287: BlockingRecv error: recv got EOF from 127.22.177.2:43287 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 10:07:51 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:51 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:51.970984 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:51.965550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:38617)
0811 10:07:51.965732 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:51.965765 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:51.965840 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:51.966030 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:51.966031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:51.966259 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:51.966311 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:51.966313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.967084 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.967085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:51.967692 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:51.967695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.968130 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.968131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:51.968194 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:51.968727 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:51.968732 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:51.970903 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
W20260811 10:07:51.971133 759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:51.965731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:38617 (local address 127.22.177.62:37063)
0811 10:07:51.965808 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:51.965811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:51.965822 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:07:51.965926 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:51.965927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:51.965947 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:51.966021 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:51.966403 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.966988 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.967797 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:51.968040 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:51.968128 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:51.971044 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:38617: BlockingRecv error: recv got EOF from 127.22.177.3:38617 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 10:07:52 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:52 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:52.670077 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:52.664497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:44127)
0811 10:07:52.664726 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:52.664741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:52.664806 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:52.665011 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:52.665012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:52.665201 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:52.665254 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:52.665256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.666001 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.666002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:52.666598 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:52.666600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.666999 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.667001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:52.667077 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:52.667734 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:52.667740 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:52.670000 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
W20260811 10:07:52.670252 763 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:52.664647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:44127 (local address 127.22.177.62:37063)
0811 10:07:52.664844 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:52.664847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:52.664856 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:52.664885 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:52.664886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:52.664897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:52.664980 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:52.665341 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.665900 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.666697 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.666918 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.666974 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:52.670160 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:44127: BlockingRecv error: recv got EOF from 127.22.177.1:44127 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
Aug 11 10:07:52 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:52 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:52.809978 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:52.804451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:59749)
0811 10:07:52.804604 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:52.804618 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:52.804718 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:52.804889 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:52.804890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:52.805067 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:52.805120 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:52.805121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.805920 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.805921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:52.806461 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:52.806463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.806863 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.806864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:52.806928 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:52.807537 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:52.807543 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:52.809916 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
W20260811 10:07:52.810142 763 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:52.804564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:59749 (local address 127.22.177.62:37063)
0811 10:07:52.804618 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:52.804620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:52.804715 (+ 95us) server_negotiation.cc:407] Connection header received
0811 10:07:52.804792 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:52.804793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:52.804807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:52.804862 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:52.805259 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.805824 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.806561 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.806757 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.806823 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:52.810056 (+ 3233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:59749: BlockingRecv error: recv got EOF from 127.22.177.2:59749 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 10:07:52 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:52 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:52.977419 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:52.971555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:35731)
0811 10:07:52.971772 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:52.971784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:52.971876 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:52.972046 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:52.972048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:52.972263 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:52.972314 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:52.972316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.973035 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.973036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:52.973642 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:52.973645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.974107 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.974108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:52.974176 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:52.974824 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:52.974831 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:52.977305 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
W20260811 10:07:52.977564 763 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:52.971779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:35731 (local address 127.22.177.62:37063)
0811 10:07:52.971868 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:52.971870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:52.971882 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:52.971938 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:52.971939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:52.971956 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:52.972026 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:52.972416 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.972929 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.973747 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:52.974017 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:52.974095 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:52.977500 (+ 3405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:35731: BlockingRecv error: recv got EOF from 127.22.177.3:35731 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 10:07:53 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:53 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:53.676584 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:53.670613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:43861)
0811 10:07:53.670797 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:53.670808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:53.670873 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:53.671126 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:53.671128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:53.671380 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:53.671454 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:53.671456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.672418 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.672419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:53.672965 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:53.672967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.673389 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.673390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:53.673443 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:53.674007 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:53.674012 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:53.676520 (+ 2508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
W20260811 10:07:53.676820 767 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:53.670775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:43861 (local address 127.22.177.62:37063)
0811 10:07:53.670956 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:53.670959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:53.670969 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:53.670996 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:53.670997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:53.671011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:53.671093 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:53.671537 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.672316 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.673047 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.673258 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.673371 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:53.676739 (+ 3368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:43861: BlockingRecv error: recv got EOF from 127.22.177.1:43861 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
Aug 11 10:07:53 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:53 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:53.816859 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:53.810603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:54361)
0811 10:07:53.810842 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:53.810855 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:53.810937 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:53.811070 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:53.811071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:53.811259 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:53.811312 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:53.811314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.812357 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.812359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:53.813252 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:53.813257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.813697 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.813698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:53.813764 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:53.814406 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:53.814411 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:53.816768 (+ 2357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":79,"threads_started":1}
W20260811 10:07:53.817037 767 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:53.810799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:54361 (local address 127.22.177.62:37063)
0811 10:07:53.810866 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:53.810869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:53.810916 (+ 47us) server_negotiation.cc:407] Connection header received
0811 10:07:53.810994 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:53.810995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:53.811008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:53.811066 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:53.811409 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.812196 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.813364 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.813578 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.813668 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:53.816927 (+ 3259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:54361: BlockingRecv error: recv got EOF from 127.22.177.2:54361 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 10:07:53 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:53 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:53.983832 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:53.978060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:42005)
0811 10:07:53.978337 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:53.978352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:53.978507 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:53.978714 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:53.978716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:53.978925 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:53.978975 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:53.978977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.979789 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.979790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:53.980380 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:53.980383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.980834 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.980835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:53.980902 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:53.981446 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:53.981452 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:53.983734 (+ 2282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":92,"threads_started":1}
W20260811 10:07:53.983906 767 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:53.978195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:42005 (local address 127.22.177.62:37063)
0811 10:07:53.978315 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:53.978318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:53.978458 (+ 140us) server_negotiation.cc:407] Connection header received
0811 10:07:53.978601 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:53.978602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:53.978621 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:53.978704 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:53.979080 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.979671 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.980489 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:53.980707 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:53.980816 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:53.983854 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:42005: BlockingRecv error: recv got EOF from 127.22.177.3:42005 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 10:07:54 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:54 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:54.682976 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:54.677148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:41965)
0811 10:07:54.677334 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:54.677347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:54.677424 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:54.677719 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:54.677720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:54.677904 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:54.677956 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:54.677958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.678761 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.678762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:54.679392 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:54.679395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.679823 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.679824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:54.679879 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:54.680474 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:54.680479 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:54.682904 (+ 2425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
W20260811 10:07:54.683152 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:54.677345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:41965 (local address 127.22.177.62:37063)
0811 10:07:54.677502 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:54.677504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:54.677513 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:54.677539 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:54.677539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:54.677551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:54.677703 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:54.678075 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.678653 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.679504 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.679736 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.679796 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:54.683044 (+ 3248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:41965: BlockingRecv error: recv got EOF from 127.22.177.1:41965 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
Aug 11 10:07:54 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:54 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:54.823982 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:54.817574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:55549)
0811 10:07:54.817846 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:54.817858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:54.817937 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:54.818231 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:54.818232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:54.818489 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:54.818541 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:54.818543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.819393 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.819394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:54.820052 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:54.820056 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.820710 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.820712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:54.820808 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:54.821437 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:54.821442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:54.823900 (+ 2458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
W20260811 10:07:54.824159 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:54.817726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:55549 (local address 127.22.177.62:37063)
0811 10:07:54.817795 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:54.817797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:54.817991 (+ 194us) server_negotiation.cc:407] Connection header received
0811 10:07:54.818032 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:54.818033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:54.818048 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:54.818172 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:54.818709 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.819279 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.820271 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.820592 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.820671 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:54.824040 (+ 3369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:55549: BlockingRecv error: recv got EOF from 127.22.177.2:55549 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 11 10:07:54 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:54 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:54.990404 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:54.984516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:57861)
0811 10:07:54.984679 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:54.984692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:54.984807 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:54.985003 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:54.985004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:54.985313 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:54.985455 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:54.985459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.986314 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.986317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:54.986906 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:54.986908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.987340 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.987341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:54.987408 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:54.988101 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:54.988107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:54.990327 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
W20260811 10:07:54.990752 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:54.984686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:57861 (local address 127.22.177.62:37063)
0811 10:07:54.984795 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:54.984797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:54.984809 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:07:54.984879 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:54.984880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:54.984897 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:54.984978 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:54.985622 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.986200 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.987018 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:54.987242 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:54.987335 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:54.990675 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:57861: BlockingRecv error: recv got EOF from 127.22.177.3:57861 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 10:07:55 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:55 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:55.694499 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:55.683682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:51953)
0811 10:07:55.683919 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:55.683936 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:55.684001 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:55.684181 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:55.684182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:55.684399 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:55.684490 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:55.684492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.685687 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.685688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:55.686246 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:55.686250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.686773 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.686775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:55.686827 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:55.692002 (+ 5175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:55.692009 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:55.694411 (+ 2402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":176,"threads_started":1}
W20260811 10:07:55.694653 775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:55.683846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:51953 (local address 127.22.177.62:37063)
0811 10:07:55.684008 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:55.684010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:55.684054 (+ 44us) server_negotiation.cc:407] Connection header received
0811 10:07:55.684081 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:55.684081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:55.684092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:55.684149 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:55.684739 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.685540 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.686398 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.686628 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.686699 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:55.694548 (+ 7849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:51953: BlockingRecv error: recv got EOF from 127.22.177.1:51953 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
Aug 11 10:07:55 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:55 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:55.830164 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:55.824682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:46929)
0811 10:07:55.824842 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:55.824860 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:55.824913 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:55.825102 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:55.825103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:55.825289 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:55.825342 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:55.825343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.826196 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.826197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:55.826837 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:55.826840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.827282 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.827283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:55.827340 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:55.827879 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:55.827884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:55.830055 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
W20260811 10:07:55.830444 775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:55.824839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:46929 (local address 127.22.177.62:37063)
0811 10:07:55.824912 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:55.824914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:55.824961 (+ 47us) server_negotiation.cc:407] Connection header received
0811 10:07:55.824990 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:55.824991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:55.825001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:55.825052 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:55.825472 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.826081 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.826952 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.827177 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.827254 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:55.830276 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:46929: BlockingRecv error: recv got EOF from 127.22.177.2:46929 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 10:07:55 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:55 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:55.996335 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:55.991131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:54155)
0811 10:07:55.991318 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:55.991389 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:55.991462 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:55.991643 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:55.991644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:55.991856 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:55.991905 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:55.991907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.992634 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.992635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:55.993175 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:55.993177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.993615 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.993616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:55.993668 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:55.994231 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:55.994236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:55.996261 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
W20260811 10:07:55.996508 775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:55.991255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:54155 (local address 127.22.177.62:37063)
0811 10:07:55.991314 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:55.991316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:55.991466 (+ 150us) server_negotiation.cc:407] Connection header received
0811 10:07:55.991535 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:55.991535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:55.991553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:55.991612 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:55.992015 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.992542 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.993297 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:55.993491 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:55.993554 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:55.996401 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:54155: BlockingRecv error: recv got EOF from 127.22.177.3:54155 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 10:07:56 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:56 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:56.700814 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:56.695184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:36261)
0811 10:07:56.695385 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:56.695401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:56.695477 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:56.695659 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:56.695660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:56.695881 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:56.695929 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:56.695931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.696657 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.696658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:56.697229 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:56.697231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.697893 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.697895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:56.697969 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:56.698670 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:56.698675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:56.700716 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
W20260811 10:07:56.700934 779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:56.695255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:36261 (local address 127.22.177.62:37063)
0811 10:07:56.695517 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:56.695519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:56.695529 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:56.695556 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:56.695556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:56.695573 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:56.695636 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:56.695999 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.696566 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.697390 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.697765 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.697846 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:56.700856 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:36261: BlockingRecv error: recv got EOF from 127.22.177.1:36261 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":194,"threads_started":1}
Aug 11 10:07:56 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:56 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:56.837067 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:56.831033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:56483)
0811 10:07:56.831268 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:56.831281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:56.831362 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:56.831540 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:56.831541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:56.831745 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:56.831796 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:56.831797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.832502 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.832504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:56.833183 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:56.833186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.833573 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.833574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:56.833672 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:56.834250 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:56.834255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:56.836930 (+ 2675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":119,"threads_started":1}
W20260811 10:07:56.837107 779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:56.831158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:56483 (local address 127.22.177.62:37063)
0811 10:07:56.831251 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:56.831253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:56.831394 (+ 141us) server_negotiation.cc:407] Connection header received
0811 10:07:56.831433 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:56.831434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:56.831456 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:56.831534 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:56.831895 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.832406 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.833283 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.833490 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.833573 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:56.837036 (+ 3463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:56483: BlockingRecv error: recv got EOF from 127.22.177.2:56483 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 10:07:56 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:57 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:57.002573 781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:56.996951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:59331)
0811 10:07:56.997111 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:56.997126 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:56.997207 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:56.997380 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:56.997381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:56.997575 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:56.997680 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:56.997682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.998456 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.998458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:56.998989 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:56.998991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.999377 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.999378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:56.999441 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:56.999970 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:56.999975 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:57.002456 (+ 2481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
W20260811 10:07:57.002648 779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:56.997080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:59331 (local address 127.22.177.62:37063)
0811 10:07:56.997131 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:56.997133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:56.997201 (+ 68us) server_negotiation.cc:407] Connection header received
0811 10:07:56.997274 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:56.997275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:56.997291 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:56.997370 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:56.997811 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.998347 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.999083 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:56.999288 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:56.999377 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:57.002570 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:59331: BlockingRecv error: recv got EOF from 127.22.177.3:59331 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 11 10:07:57 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:57 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:57.707069 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:57.701447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:55463)
0811 10:07:57.701658 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:57.701669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:57.701724 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:57.701950 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:57.701951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:57.702142 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:57.702191 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:57.702192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.702967 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.702968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:57.703541 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:57.703543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.704004 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.704006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:57.704068 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:57.704637 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:57.704642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:57.706986 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":112,"threads_started":1}
W20260811 10:07:57.707388 783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:57.701570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:55463 (local address 127.22.177.62:37063)
0811 10:07:57.701748 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:57.701750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:57.701760 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:57.701784 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:57.701785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:57.701802 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:57.701873 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:57.702324 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.702874 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.703639 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.703895 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.703956 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:57.707134 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:55463: BlockingRecv error: recv got EOF from 127.22.177.1:55463 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":121,"threads_started":1}
Aug 11 10:07:57 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:57 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:57.843495 784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:57.837872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:54839)
0811 10:07:57.838064 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:57.838077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:57.838177 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:57.838344 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:57.838345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:57.838541 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:57.838591 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:57.838593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.839289 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.839290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:57.840065 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:57.840069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.840562 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.840563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:57.840637 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:57.841204 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:57.841209 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:57.843433 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":87,"threads_started":1}
W20260811 10:07:57.843717 783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:57.837968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:54839 (local address 127.22.177.62:37063)
0811 10:07:57.838055 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:57.838057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:57.838159 (+ 102us) server_negotiation.cc:407] Connection header received
0811 10:07:57.838254 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:57.838255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:57.838268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:57.838334 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:57.838677 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.839186 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.840214 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:57.840454 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:57.840558 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:57.843614 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:54839: BlockingRecv error: recv got EOF from 127.22.177.2:54839 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 10:07:58 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:58 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:58.009059 785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:58.003215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:39075)
0811 10:07:58.003394 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:58.003405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:58.003458 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:58.003629 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:58.003630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:58.003857 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:58.003910 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:58.003912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.004698 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.004699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:58.005257 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:58.005258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.005816 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.005818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:58.005917 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:58.006464 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:58.006469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:58.008983 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
W20260811 10:07:58.009162 783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:58.003389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:39075 (local address 127.22.177.62:37063)
0811 10:07:58.003468 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:58.003470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:58.003479 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:58.003504 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:58.003504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:58.003516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:58.003577 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:58.004018 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.004570 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.005371 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.005565 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.005755 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:58.009111 (+ 3356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:39075: BlockingRecv error: recv got EOF from 127.22.177.3:39075 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 10:07:58 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:58 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:58.714713 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:58.707798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:58513)
0811 10:07:58.707992 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:58.708005 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:58.708073 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:58.708269 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:58.708270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:58.708520 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:58.708605 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:58.708607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.709643 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.709645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:58.710413 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:58.710417 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.711031 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.711032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:58.711115 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:58.711682 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:58.711689 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:58.714622 (+ 2933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":77,"threads_started":1}
W20260811 10:07:58.714829 787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:58.707948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:58513 (local address 127.22.177.62:37063)
0811 10:07:58.708089 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:58.708091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:58.708101 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:07:58.708129 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:58.708130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:58.708141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:58.708235 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:58.708719 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.709513 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.710522 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.710905 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.710997 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:58.714744 (+ 3747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:58513: BlockingRecv error: recv got EOF from 127.22.177.1:58513 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
Aug 11 10:07:58 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:58 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:58.849984 788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:58.844057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:55605)
0811 10:07:58.844213 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:58.844248 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:58.844384 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:58.844548 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:58.844548 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:58.844731 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:58.844792 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:58.844794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.845544 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.845545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:58.846278 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:58.846280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.846756 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.846757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:58.846819 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:58.847381 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:58.847385 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:58.849790 (+ 2405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
W20260811 10:07:58.850023 787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:58.844147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:55605 (local address 127.22.177.62:37063)
0811 10:07:58.844214 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:58.844216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:58.844393 (+ 177us) server_negotiation.cc:407] Connection header received
0811 10:07:58.844451 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:58.844452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:58.844467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:58.844524 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:58.844877 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.845433 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.846433 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:58.846646 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:58.846727 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:58.849973 (+ 3246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:55605: BlockingRecv error: recv got EOF from 127.22.177.2:55605 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 10:07:59 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:59 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:59.016259 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:59.009788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:47843)
0811 10:07:59.010034 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:59.010046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:59.010121 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:59.010420 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:59.010421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:59.010601 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:59.010648 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:59.010649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.011834 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.011835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:59.012580 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:59.012583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.013008 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.013009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:59.013059 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:59.013808 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:59.013815 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:59.016197 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":116,"threads_started":1}
W20260811 10:07:59.016549 787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:59.009976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:47843 (local address 127.22.177.62:37063)
0811 10:07:59.010047 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:59.010050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:59.010096 (+ 46us) server_negotiation.cc:407] Connection header received
0811 10:07:59.010270 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:59.010271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:59.010287 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:59.010385 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:59.010738 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.011588 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.012688 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.012921 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.012985 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:59.016498 (+ 3513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:47843: BlockingRecv error: recv got EOF from 127.22.177.3:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 10:07:59 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:59 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:59.721098 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:59.715518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:47595)
0811 10:07:59.715657 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:59.715668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:59.715735 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:59.715992 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:59.715994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:59.716171 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:59.716222 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:59.716224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.717029 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.717030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:59.717772 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:59.717775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.718207 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.718208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:59.718285 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:59.718944 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:59.718949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:59.721034 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260811 10:07:59.721340 791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:59.715650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:47595 (local address 127.22.177.62:37063)
0811 10:07:59.715815 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:59.715817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:59.715826 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:07:59.715848 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:59.715849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:59.715869 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:59.715947 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:59.716333 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.716894 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.717878 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.718126 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.718199 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:59.721159 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:47595: BlockingRecv error: recv got EOF from 127.22.177.1:47595 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
Aug 11 10:07:59 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:07:59 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:07:59.856482 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:07:59.850916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:56117)
0811 10:07:59.851122 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 10:07:59.851135 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:07:59.851211 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:07:59.851541 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:07:59.851542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:07:59.851726 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:07:59.851798 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:59.851800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.852539 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.852540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:59.853171 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:07:59.853174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.853674 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.853675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:07:59.853726 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:59.854350 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:07:59.854355 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:07:59.856369 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
W20260811 10:07:59.856613 791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:07:59.851043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:56117 (local address 127.22.177.62:37063)
0811 10:07:59.851121 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:07:59.851123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:07:59.851243 (+ 120us) server_negotiation.cc:407] Connection header received
0811 10:07:59.851278 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:07:59.851279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:07:59.851293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:07:59.851371 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:07:59.851906 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.852435 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.853335 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:07:59.853537 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:07:59.853692 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:07:59.856538 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:56117: BlockingRecv error: recv got EOF from 127.22.177.2:56117 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 10:08:00 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:08:00 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.3@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:08:00.026022 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:00.017005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.3:47455)
0811 10:08:00.017288 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:00.017309 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:00.017397 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:00.017633 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:00.017641 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:00.017838 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:00.017891 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:00.017893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.019100 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.019101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:00.019921 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:00.019925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.020997 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.020999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:00.021080 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:00.021941 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:00.021946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:00.025932 (+ 3986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":111,"threads_started":1}
W20260811 10:08:00.026243 791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:08:00.017223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:47455 (local address 127.22.177.62:37063)
0811 10:08:00.017324 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:00.017326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:00.017449 (+ 123us) server_negotiation.cc:407] Connection header received
0811 10:08:00.017479 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:00.017480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:00.017495 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:00.017545 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:00.018042 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.018759 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.020374 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.020704 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.020792 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:00.026148 (+ 5356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.3:47455: BlockingRecv error: recv got EOF from 127.22.177.3:47455 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 10:08:00 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:08:00 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.1@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:08:00.735505 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:00.721870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.1:49287)
0811 10:08:00.722016 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:00.722028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:00.722080 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:00.722447 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:00.722449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:00.722636 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:00.722689 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:00.722690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.723492 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.723493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:00.724060 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:00.724063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.724484 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.724485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:00.724540 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:00.732323 (+ 7783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:00.732337 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:00.735401 (+ 3064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
W20260811 10:08:00.735646 795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:08:00.721992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:49287 (local address 127.22.177.62:37063)
0811 10:08:00.722270 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:00.722272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:00.722282 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:08:00.722312 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:00.722313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:00.722325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:00.722395 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:00.722818 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.723364 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.724158 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.724392 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.724456 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:00.735572 (+ 11116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.1:49287: BlockingRecv error: recv got EOF from 127.22.177.1:49287 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":224,"threads_started":1}
Aug 11 10:08:00 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:08:00 dist-test-slave-g350 krb5kdc[32111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.177.2@KRBTEST.COM for oryx/127.22.177.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:08:00.862890 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:00.857249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37063 (local address 127.22.177.2:48037)
0811 10:08:00.857454 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:00.857471 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:00.857535 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:00.857771 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:00.857772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:00.857995 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:00.858051 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:00.858053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.858785 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.858787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:00.859389 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:00.859391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.859918 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.859920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:00.859999 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:00.860652 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:00.860659 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:00.862826 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.177.62:37063: Server oryx/127.22.177.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":138,"threads_started":1}
W20260811 10:08:00.863051 795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:08:00.857427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:48037 (local address 127.22.177.62:37063)
0811 10:08:00.857504 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:00.857506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:00.857516 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:08:00.857630 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:00.857631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:00.857645 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:00.857722 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:00.858163 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.858680 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.859487 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:00.859810 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:00.859905 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:00.862953 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.177.2:48037: BlockingRecv error: recv got EOF from 127.22.177.2:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:08:00.916290 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 32742
I20260811 10:08:00.924304 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 441
I20260811 10:08:00.933907 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 586
I20260811 10:08:00.941188 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 32668
2026-08-11T10:08:00Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17725 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T10:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:08:00Z Disabled control of system clock
I20260811 10:08:00.978578 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40463
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 10:08:01.075101 803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:01.075344 803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:01.075368 803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:01.076718 803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:08:01.076774 803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:01.076787 803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:01.076800 803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:08:01.076813 803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:08:01.078426 803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:01.078756 803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:01.079118 803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:01.085348 809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.085474 808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.085348 811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:01.085685 803 server_base.cc:1034] running on GCE node
I20260811 10:08:01.086020 803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:01.086515 803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:01.087684 803 hybrid_clock.cc:648] HybridClock initialized: now 1786442881087648 us; error 58 us; skew 500 ppm
I20260811 10:08:01.089121 803 webserver.cc:460] Webserver started at http://127.22.177.62:40431/ using document root <none> and password file <none>
I20260811 10:08:01.089486 803 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:01.089555 803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:01.089780 803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:01.090921 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "cfeffbf429774bf386d2c31ce6c510ff"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "30c7f6b76a58bd7e3783259bf3ef3f49"
server_key_iv: "b4bdc297db361ea12e014e8d918e2d04"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.091303 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "cfeffbf429774bf386d2c31ce6c510ff"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "30c7f6b76a58bd7e3783259bf3ef3f49"
server_key_iv: "b4bdc297db361ea12e014e8d918e2d04"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.092852 803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:08:01.093919 819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.094201 803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:01.094334 803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "cfeffbf429774bf386d2c31ce6c510ff"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "30c7f6b76a58bd7e3783259bf3ef3f49"
server_key_iv: "b4bdc297db361ea12e014e8d918e2d04"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.094465 803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:01.109200 803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:01.110684 803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:01.110975 803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:01.116436 803 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40463
I20260811 10:08:01.116477 878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40463 every 8 connection(s)
I20260811 10:08:01.116884 803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:08:01.117811 879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:01.120854 879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff: Bootstrap starting.
I20260811 10:08:01.121543 879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:01.121920 879 log.cc:824] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:01.122881 879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff: No bootstrap required, opened a new log
I20260811 10:08:01.124774 879 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeffbf429774bf386d2c31ce6c510ff" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40463 } }
I20260811 10:08:01.125026 879 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:01.125069 879 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfeffbf429774bf386d2c31ce6c510ff, State: Initialized, Role: FOLLOWER
I20260811 10:08:01.125154 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 803
I20260811 10:08:01.125239 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:08:01.125305 879 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeffbf429774bf386d2c31ce6c510ff" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40463 } }
I20260811 10:08:01.125402 879 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:08:01.125449 879 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:08:01.125478 23236 external_mini_cluster.cc:1428] Setting key 1aeddc9d407297541da90fb1d9c51563
I20260811 10:08:01.125506 879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:01.126340 879 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeffbf429774bf386d2c31ce6c510ff" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40463 } }
I20260811 10:08:01.126472 879 leader_election.cc:302] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfeffbf429774bf386d2c31ce6c510ff; no voters:
I20260811 10:08:01.126668 879 leader_election.cc:288] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:08:01.126696 885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:08:01.126814 885 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [term 1 LEADER]: Becoming Leader. State: Replica: cfeffbf429774bf386d2c31ce6c510ff, State: Running, Role: LEADER
I20260811 10:08:01.126965 879 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:08:01.126973 885 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeffbf429774bf386d2c31ce6c510ff" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40463 } }
I20260811 10:08:01.127607 886 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfeffbf429774bf386d2c31ce6c510ff. Latest consensus state: current_term: 1 leader_uuid: "cfeffbf429774bf386d2c31ce6c510ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeffbf429774bf386d2c31ce6c510ff" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40463 } } }
I20260811 10:08:01.127718 886 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:01.128192 887 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfeffbf429774bf386d2c31ce6c510ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeffbf429774bf386d2c31ce6c510ff" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40463 } } }
I20260811 10:08:01.128350 887 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:01.128417 894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:08:01.128998 894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:08:01.131049 894 catalog_manager.cc:1347] Generated new cluster ID: 045193f3e1474a95b96359cab73377b4
I20260811 10:08:01.131156 894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:08:01.133569 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.126132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.22.177.62:40463)
0811 10:08:01.126388 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:01.126399 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:01.126437 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:08:01.126566 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:01.126574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:01.126655 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:01.126813 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:01.127138 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.129684 (+ 2546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.130945 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.131721 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.131864 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.132757 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:01.132761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:01.132761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:01.133009 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:01.133062 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:01.133079 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:01.133153 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:01.133227 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 10:08:01.133314 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
I20260811 10:08:01.134825 894 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:08:01.135356 894 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:08:01.138882 894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfeffbf429774bf386d2c31ce6c510ff: Generated new TSK 0
I20260811 10:08:01.139106 894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:08:01.141095 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40463
--builtin_ntp_servers=127.22.177.20:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 10:08:01.236945 906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:01.237213 906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:01.237241 906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:01.238833 906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:01.238909 906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:01.238942 906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:08:01.240520 906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:01.240823 906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:01.241191 906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:01.247905 912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.247915 914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.247905 911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:01.248431 906 server_base.cc:1034] running on GCE node
I20260811 10:08:01.248677 906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:01.249135 906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:01.250366 906 hybrid_clock.cc:648] HybridClock initialized: now 1786442881250443 us; error 147 us; skew 500 ppm
I20260811 10:08:01.252228 906 webserver.cc:460] Webserver started at http://127.22.177.1:38221/ using document root <none> and password file <none>
I20260811 10:08:01.252558 906 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:01.252637 906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:01.252790 906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:01.254590 906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "ee59cc9a3d6e4ee79f8a4e5348ee9f40"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "d7a892d08e679c6e2315dab898e0d1d3"
server_key_iv: "a3b5b3cd276a9b702f0288a696a0a256"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.255348 906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "ee59cc9a3d6e4ee79f8a4e5348ee9f40"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "d7a892d08e679c6e2315dab898e0d1d3"
server_key_iv: "a3b5b3cd276a9b702f0288a696a0a256"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.257671 906 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:08:01.258838 920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.259133 906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:01.259266 906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "ee59cc9a3d6e4ee79f8a4e5348ee9f40"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "d7a892d08e679c6e2315dab898e0d1d3"
server_key_iv: "a3b5b3cd276a9b702f0288a696a0a256"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.259362 906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:01.271494 906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:01.271893 906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:01.272070 906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:01.272338 906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:01.272728 906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:01.272800 906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.272845 906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:01.272869 906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.279666 906 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:38381
I20260811 10:08:01.279819 1051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:38381 every 8 connection(s)
I20260811 10:08:01.280144 906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:08:01.286679 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.280955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:54597 (local address 127.22.177.62:40463)
0811 10:08:01.281025 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:01.281028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:01.281363 (+ 335us) server_negotiation.cc:407] Connection header received
0811 10:08:01.282080 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:01.282081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:01.282095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:01.282163 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:01.282874 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.283507 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.284632 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.284850 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.284921 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.286203 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:01.286205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:01.286205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:01.286210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:01.286226 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:01.286230 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:01.286283 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:01.286558 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 10:08:01.286597 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:08:01.286847 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.280961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40463 (local address 127.22.177.1:54597)
0811 10:08:01.281227 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:01.281267 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:01.281946 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:01.282218 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:01.282221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:01.282229 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:01.282779 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:01.282784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.283607 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.283608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:01.284517 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:01.284520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.284974 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.284975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:01.285121 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.285973 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:08:01.285973 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:01.286006 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:08:01.286031 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:01.286400 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:01.286415 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:01.286418 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:08:01.286507 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 10:08:01.286608 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":105,"threads_started":1}
I20260811 10:08:01.287765 1052 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40463
I20260811 10:08:01.287921 1052 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:01.288141 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 906
I20260811 10:08:01.288224 1052 heartbeater.cc:511] Master 127.22.177.62:40463 requested a full tablet report, sending...
I20260811 10:08:01.288309 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:08:01.288755 23236 external_mini_cluster.cc:1428] Setting key fd82b8faa44db644093ff092b2cafbf9
I20260811 10:08:01.290001 837 ts_manager.cc:194] Registered new tserver with Master: ee59cc9a3d6e4ee79f8a4e5348ee9f40 (127.22.177.1:38381)
I20260811 10:08:01.290050 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40463
--builtin_ntp_servers=127.22.177.20:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 10:08:01.290767 837 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.1:54597
W20260811 10:08:01.382016 1055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:01.382287 1055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:01.382328 1055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:01.383597 1055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:01.383677 1055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:01.383723 1055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:08:01.385207 1055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:01.385535 1055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:01.385907 1055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:01.391835 1063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.391918 1061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.391827 1060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:01.392028 1055 server_base.cc:1034] running on GCE node
I20260811 10:08:01.392328 1055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:01.392640 1055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:01.393853 1055 hybrid_clock.cc:648] HybridClock initialized: now 1786442881393862 us; error 62 us; skew 500 ppm
I20260811 10:08:01.395272 1055 webserver.cc:460] Webserver started at http://127.22.177.2:43031/ using document root <none> and password file <none>
I20260811 10:08:01.395565 1055 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:01.395641 1055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:01.395805 1055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:01.397011 1055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "104f958b063144ceb7817c4216523b12"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "77d5eda7ca70fcf1a1dcbc4dd896b4a3"
server_key_iv: "8c84dec163b9da5aa0b0dc6cc13a7783"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.397408 1055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "104f958b063144ceb7817c4216523b12"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "77d5eda7ca70fcf1a1dcbc4dd896b4a3"
server_key_iv: "8c84dec163b9da5aa0b0dc6cc13a7783"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.398969 1055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:01.400013 1071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.400225 1055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:01.400344 1055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "104f958b063144ceb7817c4216523b12"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "77d5eda7ca70fcf1a1dcbc4dd896b4a3"
server_key_iv: "8c84dec163b9da5aa0b0dc6cc13a7783"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.400432 1055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:01.414680 1055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:01.415095 1055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:01.415341 1055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:01.415634 1055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:01.416028 1055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:01.416090 1055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.416133 1055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:01.416158 1055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.422883 1055 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:43741
I20260811 10:08:01.422928 1196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:43741 every 8 connection(s)
I20260811 10:08:01.423327 1055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:08:01.427079 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1055
I20260811 10:08:01.427220 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:08:01.427491 23236 external_mini_cluster.cc:1428] Setting key 5dffc78de05ad6db8bf69667f2bc9e89
I20260811 10:08:01.428895 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40463
--builtin_ntp_servers=127.22.177.20:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 10:08:01.430174 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.424243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:36343 (local address 127.22.177.62:40463)
0811 10:08:01.424292 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:01.424294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:01.424613 (+ 319us) server_negotiation.cc:407] Connection header received
0811 10:08:01.425196 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:01.425197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:01.425209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:01.425285 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:01.426339 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.427008 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.428459 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.428666 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.428765 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.429865 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:01.429866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:01.429867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:01.429872 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:01.429890 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:01.429895 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:01.429970 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:01.430049 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 10:08:01.430091 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:08:01.430487 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.424242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40463 (local address 127.22.177.2:36343)
0811 10:08:01.424468 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:01.424514 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:01.425042 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:01.425454 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:01.425458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:01.425468 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:01.426178 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:01.426186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.427133 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.427134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:01.428321 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:01.428326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.428805 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.428806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:01.428971 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.429686 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:08:01.429686 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:01.429717 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:08:01.429743 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:01.429972 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:01.429979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:01.429983 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:08:01.430060 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:08:01.430176 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260811 10:08:01.431043 1197 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40463
I20260811 10:08:01.431177 1197 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:01.431434 1197 heartbeater.cc:511] Master 127.22.177.62:40463 requested a full tablet report, sending...
I20260811 10:08:01.432041 837 ts_manager.cc:194] Registered new tserver with Master: 104f958b063144ceb7817c4216523b12 (127.22.177.2:43741)
I20260811 10:08:01.432628 837 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.2:36343
W20260811 10:08:01.538252 1200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:01.538578 1200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:01.538625 1200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:01.540954 1200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:01.541054 1200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:01.541121 1200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:08:01.544183 1200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:01.544667 1200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:01.545120 1200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:01.552584 1209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.552736 1206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.552591 1207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:01.552946 1200 server_base.cc:1034] running on GCE node
I20260811 10:08:01.553323 1200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:01.553685 1200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:01.554878 1200 hybrid_clock.cc:648] HybridClock initialized: now 1786442881554863 us; error 33 us; skew 500 ppm
I20260811 10:08:01.557222 1200 webserver.cc:460] Webserver started at http://127.22.177.3:41837/ using document root <none> and password file <none>
I20260811 10:08:01.557726 1200 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:01.557829 1200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:01.558027 1200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:01.560003 1200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "75462536d2914680aa2896d7369699af"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "381f99ca4c0480d50a4f53907845b3c2"
server_key_iv: "58c394a24251b890521796148a1bfaf4"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.560575 1200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "75462536d2914680aa2896d7369699af"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "381f99ca4c0480d50a4f53907845b3c2"
server_key_iv: "58c394a24251b890521796148a1bfaf4"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.563057 1200 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260811 10:08:01.564066 1215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.564364 1200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:01.564509 1200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "75462536d2914680aa2896d7369699af"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "381f99ca4c0480d50a4f53907845b3c2"
server_key_iv: "58c394a24251b890521796148a1bfaf4"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.564607 1200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:01.594688 1200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:01.595095 1200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:01.595280 1200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:01.595583 1200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:01.596001 1200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:01.596068 1200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.596117 1200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:01.596143 1200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.603084 1200 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:33239
I20260811 10:08:01.603133 1349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:33239 every 8 connection(s)
I20260811 10:08:01.603554 1200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:08:01.607488 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1200
I20260811 10:08:01.607636 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:08:01.607847 23236 external_mini_cluster.cc:1428] Setting key 1235b3e0662eaaff206579ba526f99e8
I20260811 10:08:01.609679 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.604479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:46203 (local address 127.22.177.62:40463)
0811 10:08:01.604558 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:01.604560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:01.604862 (+ 302us) server_negotiation.cc:407] Connection header received
0811 10:08:01.605513 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:01.605514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:01.605528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:01.605581 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:01.606336 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.606870 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.607967 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.608223 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.608290 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.609373 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:01.609374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:01.609375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:01.609380 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:01.609397 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:01.609401 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:01.609452 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:01.609538 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:08:01.609571 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:08:01.609928 1352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.604471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40463 (local address 127.22.177.3:46203)
0811 10:08:01.604696 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:01.604733 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:01.605389 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:01.605697 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:01.605701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:01.605710 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:01.606215 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:01.606219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.606982 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.606983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:01.607873 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:01.607876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:01.608298 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:01.608299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:01.608489 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:01.609194 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:08:01.609195 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:01.609225 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:08:01.609250 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:01.609462 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:01.609468 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:01.609472 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:08:01.609542 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:08:01.609683 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260811 10:08:01.610394 1350 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40463
I20260811 10:08:01.610558 1350 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:01.610769 1350 heartbeater.cc:511] Master 127.22.177.62:40463 requested a full tablet report, sending...
I20260811 10:08:01.611191 837 ts_manager.cc:194] Registered new tserver with Master: 75462536d2914680aa2896d7369699af (127.22.177.3:33239)
I20260811 10:08:01.611624 837 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.3:46203
I20260811 10:08:01.620424 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:08:01.622992 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 906
W20260811 10:08:01.622979 884 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:08:01.622209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.22.177.62:40463)
0811 10:08:01.622286 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:01.622288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:01.622299 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:08:01.622440 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:01.622441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:01.622454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:01.622511 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:01.622887 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54016: BlockingRecv error: recv got EOF from 127.0.0.1:54016 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:08:01.634792 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1055
I20260811 10:08:01.642869 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1200
I20260811 10:08:01.652061 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 803
2026-08-11T10:08:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (710 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T10:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:08:01Z Disabled control of system clock
I20260811 10:08:01.688815 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40869
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 10:08:01.781929 1361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:01.782238 1361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:01.782281 1361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:01.783947 1361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:08:01.784075 1361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:01.784103 1361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:01.784127 1361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:08:01.784155 1361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:08:01.785940 1361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:01.786309 1361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:01.786687 1361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:01.793126 1367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.793140 1366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:01.793218 1369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:01.793356 1361 server_base.cc:1034] running on GCE node
I20260811 10:08:01.793825 1361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:01.794190 1361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:01.795405 1361 hybrid_clock.cc:648] HybridClock initialized: now 1786442881795378 us; error 46 us; skew 500 ppm
I20260811 10:08:01.796895 1361 webserver.cc:460] Webserver started at http://127.22.177.62:41537/ using document root <none> and password file <none>
I20260811 10:08:01.797217 1361 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:01.797309 1361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:01.797467 1361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:01.798796 1361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "7219ca07dd4b497d827081fe4558d926"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "68e377397126c216a6e0cf030657d467"
server_key_iv: "5295741dd6b327941de2295caacfe126"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.799201 1361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "7219ca07dd4b497d827081fe4558d926"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "68e377397126c216a6e0cf030657d467"
server_key_iv: "5295741dd6b327941de2295caacfe126"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.801087 1361 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260811 10:08:01.802220 1375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:01.802512 1361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:08:01.802642 1361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "7219ca07dd4b497d827081fe4558d926"
format_stamp: "Formatted at 2026-08-11 10:08:01 on dist-test-slave-g350"
server_key: "68e377397126c216a6e0cf030657d467"
server_key_iv: "5295741dd6b327941de2295caacfe126"
server_key_version: "encryptionkey@0"
I20260811 10:08:01.802745 1361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:01.806684 1361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:01.807004 1361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:01.807158 1361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:01.812690 1361 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40869
I20260811 10:08:01.812731 1432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40869 every 8 connection(s)
I20260811 10:08:01.813217 1361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:08:01.814074 1433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:01.815299 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1361
I20260811 10:08:01.815408 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:08:01.815672 23236 external_mini_cluster.cc:1428] Setting key 42c95d135b0ce83c8ccae5292c7dfe4d
I20260811 10:08:01.818775 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926: Bootstrap starting.
I20260811 10:08:01.819464 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:01.816482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.22.177.62:40869)
0811 10:08:01.816731 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:01.816742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:01.816782 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:08:01.817686 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:01.817696 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:01.817772 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:01.817959 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:01.818896 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:01.818899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:01.818900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:01.818905 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:01.818959 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:01.818972 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:01.819055 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:01.819164 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 10:08:01.819230 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260811 10:08:01.819753 1433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:01.820062 1433 log.cc:824] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:01.820909 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926: No bootstrap required, opened a new log
I20260811 10:08:01.823110 1433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7219ca07dd4b497d827081fe4558d926" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40869 } }
I20260811 10:08:01.823280 1433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:01.823305 1433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7219ca07dd4b497d827081fe4558d926, State: Initialized, Role: FOLLOWER
I20260811 10:08:01.823468 1433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7219ca07dd4b497d827081fe4558d926" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40869 } }
I20260811 10:08:01.823596 1433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:08:01.823624 1433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:08:01.823666 1433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:01.824314 1433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7219ca07dd4b497d827081fe4558d926" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40869 } }
I20260811 10:08:01.824419 1433 leader_election.cc:302] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7219ca07dd4b497d827081fe4558d926; no voters:
I20260811 10:08:01.824604 1433 leader_election.cc:288] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:08:01.825062 1438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:08:01.825083 1433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:08:01.825273 1438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [term 1 LEADER]: Becoming Leader. State: Replica: 7219ca07dd4b497d827081fe4558d926, State: Running, Role: LEADER
I20260811 10:08:01.825388 1438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7219ca07dd4b497d827081fe4558d926" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40869 } }
I20260811 10:08:01.825819 1438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7219ca07dd4b497d827081fe4558d926. Latest consensus state: current_term: 1 leader_uuid: "7219ca07dd4b497d827081fe4558d926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7219ca07dd4b497d827081fe4558d926" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40869 } } }
I20260811 10:08:01.825977 1438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:01.825830 1442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7219ca07dd4b497d827081fe4558d926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7219ca07dd4b497d827081fe4558d926" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40869 } } }
I20260811 10:08:01.826089 1442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926 [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:01.826189 1449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:08:01.826817 1449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:08:01.828980 1449 catalog_manager.cc:1347] Generated new cluster ID: e6da7e37349a48c0aa83630e6424772a
I20260811 10:08:01.829072 1449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:08:01.851359 1449 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:08:01.851598 1449 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:08:01.856264 1449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7219ca07dd4b497d827081fe4558d926: Generated new TSK 0
I20260811 10:08:01.856514 1449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:08:01.864616 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40869
--builtin_ntp_servers=127.22.177.20:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 10:08:02.002244 1460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:02.002527 1460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:02.002570 1460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:02.003981 1460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:02.004073 1460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:02.004122 1460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:08:02.005815 1460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:02.006255 1460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:02.006633 1460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:02.012980 1467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:02.013175 1469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:02.012986 1465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:02.013304 1460 server_base.cc:1034] running on GCE node
I20260811 10:08:02.013664 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:02.014034 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:02.015249 1460 hybrid_clock.cc:648] HybridClock initialized: now 1786442882015221 us; error 42 us; skew 500 ppm
I20260811 10:08:02.016876 1460 webserver.cc:460] Webserver started at http://127.22.177.1:36103/ using document root <none> and password file <none>
I20260811 10:08:02.017187 1460 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:02.017269 1460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:02.017448 1460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:02.019001 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "1644e037ddcf4ede9a6dee9f757aecf8"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "86a1c07368a6a5afafa32431afa2a28d"
server_key_iv: "72772f0663075c8bef1dd7b0c2abac46"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.019429 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "1644e037ddcf4ede9a6dee9f757aecf8"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "86a1c07368a6a5afafa32431afa2a28d"
server_key_iv: "72772f0663075c8bef1dd7b0c2abac46"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.021085 1460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:08:02.022363 1476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.022607 1460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:02.022740 1460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "1644e037ddcf4ede9a6dee9f757aecf8"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "86a1c07368a6a5afafa32431afa2a28d"
server_key_iv: "72772f0663075c8bef1dd7b0c2abac46"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.022867 1460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:02.027329 1460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:02.027689 1460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:02.027863 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:02.028115 1460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:02.028527 1460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:02.028585 1460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.028630 1460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:02.028656 1460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.035420 1460 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:46627
I20260811 10:08:02.035503 1601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:46627 every 8 connection(s)
I20260811 10:08:02.035904 1460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:08:02.037568 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1460
I20260811 10:08:02.037758 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:08:02.037990 23236 external_mini_cluster.cc:1428] Setting key ac8bea59428c8f8585890e1b858888a7
I20260811 10:08:02.039179 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40869
--builtin_ntp_servers=127.22.177.20:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 10:08:02.039851 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:02.036743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:47091 (local address 127.22.177.62:40869)
0811 10:08:02.036816 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:02.036818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:02.037125 (+ 307us) server_negotiation.cc:407] Connection header received
0811 10:08:02.037691 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:02.037693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:02.037701 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:02.037795 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:02.039522 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:02.039524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:02.039524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:02.039530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:02.039551 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:02.039557 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:02.039629 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:02.039722 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 10:08:02.039740 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:08:02.040004 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:02.036742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40869 (local address 127.22.177.1:47091)
0811 10:08:02.036941 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:02.036984 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:02.037531 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:02.037897 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:02.037903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:02.037921 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:02.039299 (+ 1378us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:08:02.039300 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:02.039360 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:08:02.039402 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:02.039636 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:02.039642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:02.039644 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:08:02.039709 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:08:02.039767 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260811 10:08:02.040669 1602 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40869
I20260811 10:08:02.040796 1602 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:02.040974 1602 heartbeater.cc:511] Master 127.22.177.62:40869 requested a full tablet report, sending...
I20260811 10:08:02.041692 1393 ts_manager.cc:194] Registered new tserver with Master: 1644e037ddcf4ede9a6dee9f757aecf8 (127.22.177.1:46627)
W20260811 10:08:02.131320 1605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:02.131609 1605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:02.131654 1605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:02.133054 1605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:02.133147 1605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:02.133213 1605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:08:02.134933 1605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:02.135260 1605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:02.135613 1605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:08:02.141484 1605 server_base.cc:1034] running on GCE node
W20260811 10:08:02.141427 1611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:02.141427 1612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:02.141422 1614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:02.142059 1605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:02.142362 1605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:02.143540 1605 hybrid_clock.cc:648] HybridClock initialized: now 1786442882143445 us; error 121 us; skew 500 ppm
I20260811 10:08:02.145184 1605 webserver.cc:460] Webserver started at http://127.22.177.2:35741/ using document root <none> and password file <none>
I20260811 10:08:02.145747 1605 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:02.145830 1605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:02.146018 1605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:02.147282 1605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "b88edd689cd24901a6b33f8117a037d5"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "38c4eaba813643d7fcaf2600c99a20da"
server_key_iv: "e970218e2063d9d19490c61f7c3c829d"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.147688 1605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "b88edd689cd24901a6b33f8117a037d5"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "38c4eaba813643d7fcaf2600c99a20da"
server_key_iv: "e970218e2063d9d19490c61f7c3c829d"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.149372 1605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:08:02.150584 1620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.150848 1605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:02.150952 1605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "b88edd689cd24901a6b33f8117a037d5"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "38c4eaba813643d7fcaf2600c99a20da"
server_key_iv: "e970218e2063d9d19490c61f7c3c829d"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.151013 1605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:02.155053 1605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:02.155313 1605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:02.155428 1605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:02.155594 1605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:02.155952 1605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:02.155988 1605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.156011 1605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:02.156054 1605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.163652 1605 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:44733
I20260811 10:08:02.163738 1749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:44733 every 8 connection(s)
I20260811 10:08:02.164108 1605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:08:02.166596 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1605
I20260811 10:08:02.166816 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:08:02.167037 23236 external_mini_cluster.cc:1428] Setting key 12eec090ab1c69fdd6850c2ae3b00af0
I20260811 10:08:02.168183 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40869
--builtin_ntp_servers=127.22.177.20:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 10:08:02.168179 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:02.165229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:55895 (local address 127.22.177.62:40869)
0811 10:08:02.165325 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:02.165328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:02.165541 (+ 213us) server_negotiation.cc:407] Connection header received
0811 10:08:02.166232 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:02.166234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:02.166242 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:02.166297 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:02.167846 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:02.167847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:02.167848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:02.167854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:02.167874 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:02.167879 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:02.167948 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:02.168080 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 10:08:02.168098 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 10:08:02.168437 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:02.165138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40869 (local address 127.22.177.2:55895)
0811 10:08:02.165391 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:02.165439 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:02.166096 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:02.166381 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:02.166386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:02.166397 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:02.167646 (+ 1249us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:08:02.167647 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:02.167683 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:08:02.167719 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:02.167974 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:02.167981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:02.167984 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:08:02.168083 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 10:08:02.168176 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260811 10:08:02.168998 1750 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40869
I20260811 10:08:02.169116 1750 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:02.169291 1750 heartbeater.cc:511] Master 127.22.177.62:40869 requested a full tablet report, sending...
I20260811 10:08:02.169725 1393 ts_manager.cc:194] Registered new tserver with Master: b88edd689cd24901a6b33f8117a037d5 (127.22.177.2:44733)
W20260811 10:08:02.261790 1753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:02.262038 1753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:02.262065 1753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:02.263350 1753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:02.263401 1753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:02.263430 1753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:08:02.264971 1753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:02.265230 1753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:02.265534 1753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:02.271749 1762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:02.271777 1753 server_base.cc:1034] running on GCE node
W20260811 10:08:02.271757 1758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:02.271795 1760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:02.272408 1753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:02.272727 1753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:02.273911 1753 hybrid_clock.cc:648] HybridClock initialized: now 1786442882273898 us; error 32 us; skew 500 ppm
I20260811 10:08:02.275594 1753 webserver.cc:460] Webserver started at http://127.22.177.3:33143/ using document root <none> and password file <none>
I20260811 10:08:02.275933 1753 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:02.275990 1753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:02.276149 1753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:02.277365 1753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "d71419de7e1c4ab8bc88f8bc863adee1"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "8ba10798e5a559317ee6687868fc1885"
server_key_iv: "2fc4cab8531fb00e693e846240806c21"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.277803 1753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "d71419de7e1c4ab8bc88f8bc863adee1"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "8ba10798e5a559317ee6687868fc1885"
server_key_iv: "2fc4cab8531fb00e693e846240806c21"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.279409 1753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:02.280303 1768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.280529 1753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:02.280645 1753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "d71419de7e1c4ab8bc88f8bc863adee1"
format_stamp: "Formatted at 2026-08-11 10:08:02 on dist-test-slave-g350"
server_key: "8ba10798e5a559317ee6687868fc1885"
server_key_iv: "2fc4cab8531fb00e693e846240806c21"
server_key_version: "encryptionkey@0"
I20260811 10:08:02.280726 1753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:02.284520 1753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:02.284821 1753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:02.285012 1753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:02.285252 1753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:02.285646 1753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:02.285704 1753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.285748 1753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:02.285774 1753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:02.292068 1753 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:37605
I20260811 10:08:02.292112 1895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:37605 every 8 connection(s)
I20260811 10:08:02.292527 1753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:08:02.295384 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1753
I20260811 10:08:02.295552 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:08:02.295822 23236 external_mini_cluster.cc:1428] Setting key a18b2db2cf8f731b54cc425242d632af
I20260811 10:08:02.296268 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:02.293467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:47649 (local address 127.22.177.62:40869)
0811 10:08:02.293573 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:02.293576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:02.293828 (+ 252us) server_negotiation.cc:407] Connection header received
0811 10:08:02.294388 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:02.294389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:02.294397 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:02.294458 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:02.295896 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:02.295898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:02.295899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:08:02.295904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:02.295921 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:08:02.295926 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:02.295983 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:02.296159 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 10:08:02.296178 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 10:08:02.296551 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:02.293375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40869 (local address 127.22.177.3:47649)
0811 10:08:02.293683 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:02.293733 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:02.294270 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:02.294537 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:02.294541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:02.294550 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:02.295713 (+ 1163us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:08:02.295714 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:02.295745 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:08:02.295778 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:02.296007 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:02.296016 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:02.296019 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:08:02.296121 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 10:08:02.296185 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
I20260811 10:08:02.297236 1896 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40869
I20260811 10:08:02.297370 1896 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:02.297539 1896 heartbeater.cc:511] Master 127.22.177.62:40869 requested a full tablet report, sending...
I20260811 10:08:02.298013 1393 ts_manager.cc:194] Registered new tserver with Master: d71419de7e1c4ab8bc88f8bc863adee1 (127.22.177.3:37605)
I20260811 10:08:02.308656 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 10:08:02.310863 1436 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:08:02.310488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.22.177.62:40869)
0811 10:08:02.310561 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:02.310563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:02.310572 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:08:02.310597 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:02.310598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:02.310602 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:02.310654 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:02.310794 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33898: BlockingRecv error: recv got EOF from 127.0.0.1:33898 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:08:02.310961 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1460
I20260811 10:08:02.323056 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1605
I20260811 10:08:02.332755 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1753
I20260811 10:08:02.340574 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1361
2026-08-11T10:08:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (685 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:08:02 dist-test-slave-g350 krb5kdc[1907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:08:02 dist-test-slave-g350 krb5kdc[1907](info): set up 2 sockets
Aug 11 10:08:02 dist-test-slave-g350 krb5kdc[1907](info): commencing operation
krb5kdc: starting...
W20260811 10:08:03.488093 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:08:03 dist-test-slave-g350 krb5kdc[1907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:08:03.634035 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:43543
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:43543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:03.726086 1922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:03.726364 1922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:03.726406 1922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:03.727885 1922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:08:03.727963 1922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:03.727991 1922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:03.728024 1922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:08:03.728050 1922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:08:03.729938 1922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:43543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:43543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:03.730687 1922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:03.731016 1922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:03.736876 1930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:03.736876 1929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:03.736931 1932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:03.737404 1922 server_base.cc:1034] running on GCE node
I20260811 10:08:03.737797 1922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:03.738179 1922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:03.739393 1922 hybrid_clock.cc:648] HybridClock initialized: now 1786442883739368 us; error 42 us; skew 500 ppm
Aug 11 10:08:03 dist-test-slave-g350 krb5kdc[1907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:03.742365 1922 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:08:03.742825 1922 webserver.cc:460] Webserver started at http://127.22.177.62:33107/ using document root <none> and password file <none>
I20260811 10:08:03.743094 1922 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:03.743144 1922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:03.743321 1922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:03.744545 1922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "ccdd0b8154a542a1b1da80ef8b3d5115"
format_stamp: "Formatted at 2026-08-11 10:08:03 on dist-test-slave-g350"
server_key: "7da2df9b9dc68f7394036216fb02746b"
server_key_iv: "c3ad5636f0450adf864789bbeb8e2fa9"
server_key_version: "encryptionkey@0"
I20260811 10:08:03.744941 1922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "ccdd0b8154a542a1b1da80ef8b3d5115"
format_stamp: "Formatted at 2026-08-11 10:08:03 on dist-test-slave-g350"
server_key: "7da2df9b9dc68f7394036216fb02746b"
server_key_iv: "c3ad5636f0450adf864789bbeb8e2fa9"
server_key_version: "encryptionkey@0"
I20260811 10:08:03.746639 1922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 10:08:03.747689 1939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:03.748008 1922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:03.748158 1922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ccdd0b8154a542a1b1da80ef8b3d5115"
format_stamp: "Formatted at 2026-08-11 10:08:03 on dist-test-slave-g350"
server_key: "7da2df9b9dc68f7394036216fb02746b"
server_key_iv: "c3ad5636f0450adf864789bbeb8e2fa9"
server_key_version: "encryptionkey@0"
I20260811 10:08:03.748260 1922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:03.772152 1922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:03.774933 1922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:03.775134 1922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:03.779883 1922 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:43543
I20260811 10:08:03.779944 1997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:43543 every 8 connection(s)
I20260811 10:08:03.780360 1922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:08:03.780843 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 1922
I20260811 10:08:03.780959 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:08:03.781230 23236 external_mini_cluster.cc:1428] Setting key 5788f5b1b7eca559be29483cd1285e41
I20260811 10:08:03.781987 1998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:03.785933 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: Bootstrap starting.
I20260811 10:08:03.786677 1998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:03.787055 1998 log.cc:824] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:03.787959 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: No bootstrap required, opened a new log
Aug 11 10:08:03 dist-test-slave-g350 krb5kdc[1907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:03.790568 1998 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43543 } }
I20260811 10:08:03.790768 1998 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:03.790822 1998 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdd0b8154a542a1b1da80ef8b3d5115, State: Initialized, Role: FOLLOWER
I20260811 10:08:03.790971 1998 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43543 } }
I20260811 10:08:03.791064 1998 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:08:03.791105 1998 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:08:03.791165 1998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:03.792025 1998 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43543 } }
I20260811 10:08:03.792155 1998 leader_election.cc:302] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccdd0b8154a542a1b1da80ef8b3d5115; no voters:
I20260811 10:08:03.792395 1998 leader_election.cc:288] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:08:03.792435 2004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:08:03.792555 2004 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [term 1 LEADER]: Becoming Leader. State: Replica: ccdd0b8154a542a1b1da80ef8b3d5115, State: Running, Role: LEADER
I20260811 10:08:03.792642 2004 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43543 } }
I20260811 10:08:03.792686 1998 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:08:03.793026 2004 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43543 } } }
I20260811 10:08:03.793113 2004 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:03.793254 2004 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccdd0b8154a542a1b1da80ef8b3d5115. Latest consensus state: current_term: 1 leader_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdd0b8154a542a1b1da80ef8b3d5115" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43543 } } }
I20260811 10:08:03.793335 2004 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115 [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:03.794374 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:03.781943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.22.177.62:43543)
0811 10:08:03.782214 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:03.782223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:03.782258 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:08:03.782762 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:03.782770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:03.782820 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:03.782945 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:03.783793 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:03.784823 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:03.785949 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:03.786557 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:03.786685 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:03.789772 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:03.789777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:03.789778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:03.789795 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:03.792062 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:03.792656 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:03.792661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:03.792661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:03.792829 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:03.793074 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:03.793075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:03.793075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:03.793427 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:03.793568 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:03.793925 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 10:08:03.794096 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260811 10:08:03.794610 2021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:08:03.794661 2021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:08:03.794732 2010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:08:03.794901 2010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:08:03.796576 2010 catalog_manager.cc:1347] Generated new cluster ID: 3fd072c17ea149e486ab0be0bc06edf9
I20260811 10:08:03.796634 2010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:08:03.812371 2010 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:08:03.813069 2010 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:08:03.825737 2010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: Generated new TSK 0
I20260811 10:08:03.826043 2010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:08:03.866506 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43543
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:03.958945 2026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:03.959216 2026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:03.959260 2026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:03.960508 2026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:03.960587 2026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:03.960633 2026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:08:03.962327 2026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:03.962675 2026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:03.962971 2026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:03.969237 2034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:03.969347 2026 server_base.cc:1034] running on GCE node
W20260811 10:08:03.969204 2031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:03.969241 2032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:03.969740 2026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:03.970018 2026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:03.971235 2026 hybrid_clock.cc:648] HybridClock initialized: now 1786442883971212 us; error 38 us; skew 500 ppm
Aug 11 10:08:03 dist-test-slave-g350 krb5kdc[1907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:03.974205 2026 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:08:03.974893 2026 webserver.cc:460] Webserver started at http://127.22.177.1:41933/ using document root <none> and password file <none>
I20260811 10:08:03.975281 2026 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:03.975369 2026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:03.975526 2026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:03.976742 2026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "0df623b4f6154f1ba120eec41ecdaa0a"
format_stamp: "Formatted at 2026-08-11 10:08:03 on dist-test-slave-g350"
server_key: "b77963ad93ff85054d913cd3dc77cb27"
server_key_iv: "bb87f40404eef2484a91aabe445c96b1"
server_key_version: "encryptionkey@0"
I20260811 10:08:03.977141 2026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "0df623b4f6154f1ba120eec41ecdaa0a"
format_stamp: "Formatted at 2026-08-11 10:08:03 on dist-test-slave-g350"
server_key: "b77963ad93ff85054d913cd3dc77cb27"
server_key_iv: "bb87f40404eef2484a91aabe445c96b1"
server_key_version: "encryptionkey@0"
I20260811 10:08:03.978745 2026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 10:08:03.979734 2041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:03.980005 2026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:08:03.980134 2026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "0df623b4f6154f1ba120eec41ecdaa0a"
format_stamp: "Formatted at 2026-08-11 10:08:03 on dist-test-slave-g350"
server_key: "b77963ad93ff85054d913cd3dc77cb27"
server_key_iv: "bb87f40404eef2484a91aabe445c96b1"
server_key_version: "encryptionkey@0"
I20260811 10:08:03.980218 2026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:04.002759 2026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:04.005980 2026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:04.006220 2026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:04.006613 2026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:04.007005 2026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:04.007068 2026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.007105 2026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:04.007133 2026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.014647 2026 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:34887
I20260811 10:08:04.014736 2166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:34887 every 8 connection(s)
I20260811 10:08:04.015328 2026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[1907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:04.023952 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 2026
I20260811 10:08:04.024071 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:08:04.024379 23236 external_mini_cluster.cc:1428] Setting key 9d534987b9d5af2f67bb16f9f65de10d
I20260811 10:08:04.028110 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.016553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:44581 (local address 127.22.177.62:43543)
0811 10:08:04.016632 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.016634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.016860 (+ 226us) server_negotiation.cc:407] Connection header received
0811 10:08:04.017421 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.017422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.017434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.017512 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:04.018863 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.019427 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.020692 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.020965 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.021040 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.023894 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:04.023897 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:04.023897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:04.023908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:04.026200 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.026696 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.026697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.026698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.026767 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.027100 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.027101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.027102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.027283 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:04.027362 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.027928 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.028025 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:08:04.028389 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.016457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:43543 (local address 127.22.177.1:44581)
0811 10:08:04.016648 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.016702 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.017259 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.017653 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.017657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.018440 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:04.018703 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.018710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.019560 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.019561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.020589 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.020592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.021095 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.021096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.021300 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.022017 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:04.022027 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:04.023723 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:04.026333 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:04.026341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:04.026344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:04.026615 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:04.026872 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:04.026873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:04.026873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:04.027016 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:04.027473 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:04.027483 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:04.027691 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.027876 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.028096 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260811 10:08:04.029073 2167 heartbeater.cc:347] Connected to a master server at 127.22.177.62:43543
I20260811 10:08:04.029209 2167 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:04.029430 2167 heartbeater.cc:511] Master 127.22.177.62:43543 requested a full tablet report, sending...
I20260811 10:08:04.030243 1956 ts_manager.cc:194] Registered new tserver with Master: 0df623b4f6154f1ba120eec41ecdaa0a (127.22.177.1:34887)
I20260811 10:08:04.031004 1956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:44581
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:08:04.058459 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43543
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:04.146679 2172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:04.146957 2172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:04.147001 2172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:04.148304 2172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:04.148393 2172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:04.148439 2172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:08:04.150251 2172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:04.150578 2172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:04.150846 2172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:04.157117 2180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:04.157109 2177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:04.157250 2172 server_base.cc:1034] running on GCE node
W20260811 10:08:04.157112 2178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:04.157737 2172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:04.158028 2172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:04.159300 2172 hybrid_clock.cc:648] HybridClock initialized: now 1786442884159202 us; error 42 us; skew 500 ppm
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[1907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:04.162201 2172 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:08:04.162746 2172 webserver.cc:460] Webserver started at http://127.22.177.2:35483/ using document root <none> and password file <none>
I20260811 10:08:04.163026 2172 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:04.163075 2172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:04.163213 2172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:04.164418 2172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "f0a60b869ecb47b381a74e7a54821733"
format_stamp: "Formatted at 2026-08-11 10:08:04 on dist-test-slave-g350"
server_key: "ad2b8c141f89468b21c2cd3c1d1df191"
server_key_iv: "5a06e04e3d778593fbcf3c3c63ebb61a"
server_key_version: "encryptionkey@0"
I20260811 10:08:04.164830 2172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "f0a60b869ecb47b381a74e7a54821733"
format_stamp: "Formatted at 2026-08-11 10:08:04 on dist-test-slave-g350"
server_key: "ad2b8c141f89468b21c2cd3c1d1df191"
server_key_iv: "5a06e04e3d778593fbcf3c3c63ebb61a"
server_key_version: "encryptionkey@0"
I20260811 10:08:04.166491 2172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:08:04.167537 2187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.167819 2172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:08:04.167977 2172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "f0a60b869ecb47b381a74e7a54821733"
format_stamp: "Formatted at 2026-08-11 10:08:04 on dist-test-slave-g350"
server_key: "ad2b8c141f89468b21c2cd3c1d1df191"
server_key_iv: "5a06e04e3d778593fbcf3c3c63ebb61a"
server_key_version: "encryptionkey@0"
I20260811 10:08:04.168165 2172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:04.187492 2172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:04.191484 2172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:04.191718 2172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:04.192154 2172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:04.192696 2172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:04.192764 2172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.192798 2172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:04.192816 2172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.199733 2172 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:42813
I20260811 10:08:04.199784 2312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:42813 every 8 connection(s)
I20260811 10:08:04.200151 2172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:08:04.205340 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 2172
I20260811 10:08:04.205540 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:08:04.205869 23236 external_mini_cluster.cc:1428] Setting key 8701a63e35a36ca10be8e7163737dbbb
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[1907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:04.215394 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.201342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:49353 (local address 127.22.177.62:43543)
0811 10:08:04.201445 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.201447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.201466 (+ 19us) server_negotiation.cc:407] Connection header received
0811 10:08:04.202297 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.202299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.202311 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.202375 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:04.204225 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.204776 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.207001 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.207273 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.207448 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.211660 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:04.211667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:04.211668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:04.211677 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:04.213585 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.214214 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.214215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.214216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.214286 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.214627 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.214628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.214629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.214773 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:04.214861 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.215178 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.215290 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 10:08:04.215636 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.201120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:43543 (local address 127.22.177.2:49353)
0811 10:08:04.201325 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.201384 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.202155 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.202749 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.202756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.203679 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:04.204002 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.204012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.205122 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.205124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.206819 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.206827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.207600 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.207607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.207850 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.208827 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:04.208838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:04.211347 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:04.213777 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:04.213794 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:04.213798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:04.214114 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:04.214416 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:04.214417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:04.214417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:04.214538 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:04.214857 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:04.214867 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:04.215026 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.215143 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.215357 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
I20260811 10:08:04.216703 2313 heartbeater.cc:347] Connected to a master server at 127.22.177.62:43543
I20260811 10:08:04.216849 2313 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:04.217090 2313 heartbeater.cc:511] Master 127.22.177.62:43543 requested a full tablet report, sending...
I20260811 10:08:04.217677 1956 ts_manager.cc:194] Registered new tserver with Master: f0a60b869ecb47b381a74e7a54821733 (127.22.177.2:42813)
I20260811 10:08:04.218353 1956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:49353
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:08:04.253180 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43543
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:04.352562 2319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:04.352861 2319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:04.352911 2319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:04.354475 2319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:04.354538 2319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:04.354573 2319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:08:04.356316 2319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:04.356555 2319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:04.356762 2319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:04.362157 2324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:04.362339 2327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:04.362388 2325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:04.362582 2319 server_base.cc:1034] running on GCE node
I20260811 10:08:04.362828 2319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:04.363101 2319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:04.364270 2319 hybrid_clock.cc:648] HybridClock initialized: now 1786442884364239 us; error 73 us; skew 500 ppm
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[1907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:04.367321 2319 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:08:04.367784 2319 webserver.cc:460] Webserver started at http://127.22.177.3:41041/ using document root <none> and password file <none>
I20260811 10:08:04.367986 2319 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:04.368029 2319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:04.368121 2319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:08:04.369222 2319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "07453e1b595a4661b810289112054636"
format_stamp: "Formatted at 2026-08-11 10:08:04 on dist-test-slave-g350"
server_key: "ac531dc49c41499a68ccc2dab4e8f8a8"
server_key_iv: "1c5b49f9eed352001066a0dcd2da974b"
server_key_version: "encryptionkey@0"
I20260811 10:08:04.369545 2319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "07453e1b595a4661b810289112054636"
format_stamp: "Formatted at 2026-08-11 10:08:04 on dist-test-slave-g350"
server_key: "ac531dc49c41499a68ccc2dab4e8f8a8"
server_key_iv: "1c5b49f9eed352001066a0dcd2da974b"
server_key_version: "encryptionkey@0"
I20260811 10:08:04.371129 2319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:04.372262 2337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.372601 2319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:04.372716 2319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "07453e1b595a4661b810289112054636"
format_stamp: "Formatted at 2026-08-11 10:08:04 on dist-test-slave-g350"
server_key: "ac531dc49c41499a68ccc2dab4e8f8a8"
server_key_iv: "1c5b49f9eed352001066a0dcd2da974b"
server_key_version: "encryptionkey@0"
I20260811 10:08:04.372802 2319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:04.387651 2319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:04.390069 2319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:04.390275 2319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:04.390537 2319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:04.390900 2319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:04.390957 2319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.391002 2319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:04.391033 2319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:04.397697 2319 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:36241
I20260811 10:08:04.397768 2461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:36241 every 8 connection(s)
I20260811 10:08:04.398175 2319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:08:04.400437 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 2319
I20260811 10:08:04.400532 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:08:04.400745 23236 external_mini_cluster.cc:1428] Setting key 867937eeb66b63b042e6e8f09ec2d282
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[1907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:04.409909 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.399057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:45331 (local address 127.22.177.62:43543)
0811 10:08:04.399150 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.399152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.399376 (+ 224us) server_negotiation.cc:407] Connection header received
0811 10:08:04.399885 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.399887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.399906 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.399969 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:04.401294 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.402098 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.403215 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.403459 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.403524 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.406403 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:04.406405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:04.406405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:04.406413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:04.407988 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.408597 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.408598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.408598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.408668 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.409068 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.409069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.409070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.409216 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:04.409287 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.409727 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.409828 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 10:08:04.410449 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.399002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:43543 (local address 127.22.177.3:45331)
0811 10:08:04.399200 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.399240 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.399779 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.400049 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.400053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.400851 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:04.401187 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.401195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.402213 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.402214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.403118 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.403121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.403572 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.403574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.403706 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.404488 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:04.404498 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:04.406271 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:04.408135 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:04.408172 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:04.408175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:04.408493 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:04.408790 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:04.408791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:04.408792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:04.408934 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:04.409333 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:04.409340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:04.409485 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.409978 (+ 493us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.410215 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260811 10:08:04.411157 2462 heartbeater.cc:347] Connected to a master server at 127.22.177.62:43543
I20260811 10:08:04.411301 2462 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:04.411509 2462 heartbeater.cc:511] Master 127.22.177.62:43543 requested a full tablet report, sending...
I20260811 10:08:04.412053 1956 ts_manager.cc:194] Registered new tserver with Master: 07453e1b595a4661b810289112054636 (127.22.177.3:36241)
I20260811 10:08:04.412509 1956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:45331
I20260811 10:08:04.414469 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:08:04.430545 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.416330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.22.177.62:43543)
0811 10:08:04.417342 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.417345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.417356 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:08:04.417383 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.417384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.417396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.417489 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:04.418381 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.419198 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.420368 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.420673 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.420716 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.426841 (+ 6125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:04.426843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:04.426843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:04.426865 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:04.428467 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.429088 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.429090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.429090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.429157 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:04.429545 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:04.429547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:04.429547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:04.429911 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:04.430006 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.430351 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.430471 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977}
I20260811 10:08:04.433069 1956 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:08:04.434981 1956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:08:04.442842 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.438445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.22.177.2:42813)
0811 10:08:04.438655 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.438661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.438712 (+ 51us) server_negotiation.cc:407] Connection header received
0811 10:08:04.438907 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.438910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.438927 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.438985 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:04.439399 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.440259 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.441516 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.442382 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.442459 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.442525 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.442701 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.442747 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260811 10:08:04.442853 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.438301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:42813 (local address 127.0.0.1:52592)
0811 10:08:04.438573 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.438627 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.438762 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.439101 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.439104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.439106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:04.439221 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.439223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.440366 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.440367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.441373 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.441375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.442477 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.442478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.442572 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.442587 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.442672 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.442732 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":68,"threads_started":1}
I20260811 10:08:04.443195 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.438535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:34887 (local address 127.0.0.1:49692)
0811 10:08:04.439338 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.439350 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.439407 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.439662 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.439663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.439665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:04.439711 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.439712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.441142 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.441143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.442032 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.442034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.442956 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.442957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.443025 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.443030 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.443087 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.443129 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":1,"thread_start_us":75,"threads_started":1}
I20260811 10:08:04.443234 2480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.438785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.22.177.1:34887)
0811 10:08:04.438959 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.438964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.439481 (+ 517us) server_negotiation.cc:407] Connection header received
0811 10:08:04.439533 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.439536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.439553 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.439643 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:04.439781 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.440985 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.442127 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.442794 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.442861 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.442917 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.443108 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.443151 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260811 10:08:04.443574 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.438347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:36241 (local address 127.0.0.1:44318)
0811 10:08:04.438713 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.438723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.438764 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.439232 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.439234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.439237 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:04.439301 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.439304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.440819 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.440820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.442139 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.442142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.443273 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.443274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.443360 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.443366 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.443437 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.443488 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":156,"threads_started":1}
I20260811 10:08:04.443627 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.438683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44318 (local address 127.22.177.3:36241)
0811 10:08:04.439030 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.439034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.439063 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:08:04.439115 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.439117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.439137 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.439218 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:04.439402 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.440673 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.442255 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.443110 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.443183 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.443234 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.443505 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.443556 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":111,"threads_started":1}
I20260811 10:08:04.444128 2097 tablet_service.cc:1467] Processing CreateTablet for tablet 5857355e676041dc8721eeb719a94ecb (DEFAULT_TABLE table=test-table [id=c7cf105e8542428395c0dd71845128a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:08:04.444497 2097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5857355e676041dc8721eeb719a94ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:04.446497 2390 tablet_service.cc:1467] Processing CreateTablet for tablet 5857355e676041dc8721eeb719a94ecb (DEFAULT_TABLE table=test-table [id=c7cf105e8542428395c0dd71845128a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:08:04.446900 2390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5857355e676041dc8721eeb719a94ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:04.446884 2240 tablet_service.cc:1467] Processing CreateTablet for tablet 5857355e676041dc8721eeb719a94ecb (DEFAULT_TABLE table=test-table [id=c7cf105e8542428395c0dd71845128a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:08:04.447240 2240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5857355e676041dc8721eeb719a94ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:04.448001 2481 tablet_bootstrap.cc:492] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: Bootstrap starting.
I20260811 10:08:04.448801 2481 tablet_bootstrap.cc:654] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:04.449380 2481 log.cc:824] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:04.450232 2481 tablet_bootstrap.cc:492] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: No bootstrap required, opened a new log
I20260811 10:08:04.450330 2481 ts_tablet_manager.cc:1397] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:08:04.450764 2484 tablet_bootstrap.cc:492] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Bootstrap starting.
I20260811 10:08:04.451566 2484 tablet_bootstrap.cc:654] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:04.452140 2484 log.cc:824] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:04.452605 2483 tablet_bootstrap.cc:492] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Bootstrap starting.
I20260811 10:08:04.452557 2481 raft_consensus.cc:348] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.452783 2481 raft_consensus.cc:374] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:04.452848 2481 raft_consensus.cc:729] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df623b4f6154f1ba120eec41ecdaa0a, State: Initialized, Role: FOLLOWER
I20260811 10:08:04.452991 2481 consensus_queue.cc:260] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.453365 2481 ts_tablet_manager.cc:1428] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:08:04.453413 2167 heartbeater.cc:503] Master 127.22.177.62:43543 was elected leader, sending a full tablet report...
I20260811 10:08:04.453446 2483 tablet_bootstrap.cc:654] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:04.454041 2484 tablet_bootstrap.cc:492] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: No bootstrap required, opened a new log
I20260811 10:08:04.454118 2483 log.cc:824] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:04.454139 2484 ts_tablet_manager.cc:1397] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:08:04.454854 2483 tablet_bootstrap.cc:492] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: No bootstrap required, opened a new log
I20260811 10:08:04.454908 2483 ts_tablet_manager.cc:1397] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:08:04.456195 2484 raft_consensus.cc:348] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.456250 2483 raft_consensus.cc:348] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.456370 2483 raft_consensus.cc:374] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:04.456393 2483 raft_consensus.cc:729] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07453e1b595a4661b810289112054636, State: Initialized, Role: FOLLOWER
I20260811 10:08:04.456374 2484 raft_consensus.cc:374] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:04.456425 2484 raft_consensus.cc:729] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a60b869ecb47b381a74e7a54821733, State: Initialized, Role: FOLLOWER
I20260811 10:08:04.456539 2483 consensus_queue.cc:260] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.456573 2484 consensus_queue.cc:260] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.456841 2462 heartbeater.cc:503] Master 127.22.177.62:43543 was elected leader, sending a full tablet report...
I20260811 10:08:04.456861 2483 ts_tablet_manager.cc:1428] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:08:04.456893 2313 heartbeater.cc:503] Master 127.22.177.62:43543 was elected leader, sending a full tablet report...
I20260811 10:08:04.456904 2484 ts_tablet_manager.cc:1428] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260811 10:08:04.516397 2168 tablet.cc:2369] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:08:04.584606 2489 raft_consensus.cc:482] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:08:04.584753 2489 raft_consensus.cc:504] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.585258 2489 leader_election.cc:288] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0a60b869ecb47b381a74e7a54821733 (127.22.177.2:42813), 0df623b4f6154f1ba120eec41ecdaa0a (127.22.177.1:34887)
I20260811 10:08:04.588472 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.585804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:60549 (local address 127.22.177.2:42813)
0811 10:08:04.585861 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.585864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.585877 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:08:04.585910 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.585911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.585925 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.586003 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:04.586192 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.586681 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.587687 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.588086 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.588182 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.588192 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.588354 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.588395 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:08:04.588472 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.585641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:42813 (local address 127.22.177.3:60549)
0811 10:08:04.585713 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.585727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.585779 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.586026 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.586027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.586030 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:04.586110 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.586112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.586801 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.586802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.587579 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.587581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.588179 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.588180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.588274 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.588277 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.588348 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.588393 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 10:08:04.588568 2480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.585893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:32919 (local address 127.22.177.1:34887)
0811 10:08:04.585939 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.585941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.586116 (+ 175us) server_negotiation.cc:407] Connection header received
0811 10:08:04.586190 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.586191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.586212 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.586274 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:04.586429 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.586890 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.587832 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.588201 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.588285 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.588302 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.588454 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.588496 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:08:04.588809 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.585816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:34887 (local address 127.22.177.3:32919)
0811 10:08:04.586036 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.586046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:04.586121 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:04.586278 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:04.586279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:04.586280 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:04.586353 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.586355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.586980 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.586981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.587715 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:04.587717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.588300 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.588301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:04.588374 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.588378 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:08:04.588447 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 10:08:04.588489 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":164,"threads_started":1}
I20260811 10:08:04.588845 2262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5857355e676041dc8721eeb719a94ecb" candidate_uuid: "07453e1b595a4661b810289112054636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a60b869ecb47b381a74e7a54821733" is_pre_election: true
I20260811 10:08:04.589051 2262 raft_consensus.cc:2454] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07453e1b595a4661b810289112054636 in term 0.
I20260811 10:08:04.589203 2119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5857355e676041dc8721eeb719a94ecb" candidate_uuid: "07453e1b595a4661b810289112054636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" is_pre_election: true
I20260811 10:08:04.589357 2119 raft_consensus.cc:2454] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07453e1b595a4661b810289112054636 in term 0.
I20260811 10:08:04.589331 2338 leader_election.cc:302] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07453e1b595a4661b810289112054636, f0a60b869ecb47b381a74e7a54821733; no voters:
I20260811 10:08:04.589486 2489 raft_consensus.cc:2781] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:08:04.589574 2489 raft_consensus.cc:482] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:08:04.589663 2489 raft_consensus.cc:3037] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:04.590673 2489 raft_consensus.cc:504] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.590829 2489 leader_election.cc:288] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [CANDIDATE]: Term 1 election: Requested vote from peers f0a60b869ecb47b381a74e7a54821733 (127.22.177.2:42813), 0df623b4f6154f1ba120eec41ecdaa0a (127.22.177.1:34887)
I20260811 10:08:04.591089 2262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5857355e676041dc8721eeb719a94ecb" candidate_uuid: "07453e1b595a4661b810289112054636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a60b869ecb47b381a74e7a54821733"
I20260811 10:08:04.591125 2119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5857355e676041dc8721eeb719a94ecb" candidate_uuid: "07453e1b595a4661b810289112054636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df623b4f6154f1ba120eec41ecdaa0a"
I20260811 10:08:04.591187 2262 raft_consensus.cc:3037] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:04.591235 2119 raft_consensus.cc:3037] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:04.592224 2262 raft_consensus.cc:2454] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07453e1b595a4661b810289112054636 in term 1.
I20260811 10:08:04.592391 2119 raft_consensus.cc:2454] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07453e1b595a4661b810289112054636 in term 1.
I20260811 10:08:04.592453 2338 leader_election.cc:302] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07453e1b595a4661b810289112054636, f0a60b869ecb47b381a74e7a54821733; no voters:
I20260811 10:08:04.592582 2489 raft_consensus.cc:2781] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:08:04.592813 2489 raft_consensus.cc:686] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 1 LEADER]: Becoming Leader. State: Replica: 07453e1b595a4661b810289112054636, State: Running, Role: LEADER
I20260811 10:08:04.592928 2489 consensus_queue.cc:237] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } }
I20260811 10:08:04.594058 1956 catalog_manager.cc:5495] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07453e1b595a4661b810289112054636 (127.22.177.3). New cstate: current_term: 1 leader_uuid: "07453e1b595a4661b810289112054636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07453e1b595a4661b810289112054636" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 } health_report { overall_health: UNKNOWN } } }
I20260811 10:08:04.631773 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.628906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.22.177.3:36241)
0811 10:08:04.628995 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:04.628997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:04.629007 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:08:04.629047 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:04.629048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:04.629067 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:04.629136 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:08:04.629361 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.629921 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.630756 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:04.630961 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:04.631023 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:04.631211 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:08:04.631378 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:08:04.631451 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:04.631635 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 10:08:04.631674 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 10:08:04.634308 2262 raft_consensus.cc:1261] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 1 FOLLOWER]: Refusing update from remote peer 07453e1b595a4661b810289112054636: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:08:04.634325 2119 raft_consensus.cc:1261] T 5857355e676041dc8721eeb719a94ecb P 0df623b4f6154f1ba120eec41ecdaa0a [term 1 FOLLOWER]: Refusing update from remote peer 07453e1b595a4661b810289112054636: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:08:04.634864 2492 consensus_queue.cc:1035] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a60b869ecb47b381a74e7a54821733" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 42813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:08:04.635033 2489 consensus_queue.cc:1035] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0df623b4f6154f1ba120eec41ecdaa0a" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:08:04.638106 2498 mvcc.cc:205] Tried to move back new op lower bound from 7317270055457800192 to 7317270055293571072. Current Snapshot: MvccSnapshot[applied={T|T < 7317270055457800192}]
I20260811 10:08:04.640633 1955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:08:04.640760 1955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:08:04.641741 1955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccdd0b8154a542a1b1da80ef8b3d5115: Sending DeleteTablet for 3 replicas of tablet 5857355e676041dc8721eeb719a94ecb
I20260811 10:08:04.642077 2240 tablet_service.cc:1514] Processing DeleteTablet for tablet 5857355e676041dc8721eeb719a94ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:08:04 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:52592
I20260811 10:08:04.642138 2097 tablet_service.cc:1514] Processing DeleteTablet for tablet 5857355e676041dc8721eeb719a94ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:08:04 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:49692
I20260811 10:08:04.642333 2390 tablet_service.cc:1514] Processing DeleteTablet for tablet 5857355e676041dc8721eeb719a94ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:08:04 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:44318
I20260811 10:08:04.642582 2509 tablet_replica.cc:330] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: stopping tablet replica
I20260811 10:08:04.642685 2507 tablet_replica.cc:330] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: stopping tablet replica
I20260811 10:08:04.642711 2509 raft_consensus.cc:2229] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:08:04.642792 2507 raft_consensus.cc:2229] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:08:04.642843 2509 raft_consensus.cc:2258] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:08:04.642902 2507 raft_consensus.cc:2258] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:08:04.643007 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 2026
I20260811 10:08:04.646376 2507 ts_tablet_manager.cc:1905] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:08:04.646917 2509 ts_tablet_manager.cc:1905] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:08:04.648811 2507 ts_tablet_manager.cc:1918] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:08:04.648916 2507 log.cc:1196] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/5857355e676041dc8721eeb719a94ecb
I20260811 10:08:04.649011 2509 ts_tablet_manager.cc:1918] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:08:04.649099 2509 log.cc:1196] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/5857355e676041dc8721eeb719a94ecb
I20260811 10:08:04.649290 2507 ts_tablet_manager.cc:1939] T 5857355e676041dc8721eeb719a94ecb P f0a60b869ecb47b381a74e7a54821733: Deleting consensus metadata
I20260811 10:08:04.649452 2509 ts_tablet_manager.cc:1939] T 5857355e676041dc8721eeb719a94ecb P 07453e1b595a4661b810289112054636: Deleting consensus metadata
I20260811 10:08:04.649863 1940 catalog_manager.cc:4841] TS f0a60b869ecb47b381a74e7a54821733 (127.22.177.2:42813): tablet 5857355e676041dc8721eeb719a94ecb (table test-table [id=c7cf105e8542428395c0dd71845128a1]) successfully deleted
I20260811 10:08:04.650233 1943 catalog_manager.cc:4841] TS 07453e1b595a4661b810289112054636 (127.22.177.3:36241): tablet 5857355e676041dc8721eeb719a94ecb (table test-table [id=c7cf105e8542428395c0dd71845128a1]) successfully deleted
W20260811 10:08:04.654778 1940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:34887 (error 108)
I20260811 10:08:04.655001 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 2172
I20260811 10:08:04.657799 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:04.657537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:34887 (local address 127.0.0.1:49694)
0811 10:08:04.657668 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:04.657739 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:34887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 10:08:04.658006 1940 catalog_manager.cc:4584] TS 0df623b4f6154f1ba120eec41ecdaa0a (127.22.177.1:34887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5857355e676041dc8721eeb719a94ecb: Network error: Client connection negotiation failed: client connection to 127.22.177.1:34887: connect: Connection refused (error 111)
I20260811 10:08:04.667217 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 2319
I20260811 10:08:04.675522 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 1922
2026-08-11T10:08:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2336 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[2513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[2513](info): set up 2 sockets
Aug 11 10:08:04 dist-test-slave-g350 krb5kdc[2513](info): commencing operation
krb5kdc: starting...
W20260811 10:08:05.825045 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:08:05 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:08:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/rangeradmin_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/rangeradmin_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/rangerlookup_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/rangerlookup_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
I20260811 10:08:06.038282 23236 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2wzoa9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/postgres -l logfile start
2026-08-11 10:08:09.443 UTC [2545] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 10:08:09.443 UTC [2545] LOG: listening on IPv4 address "127.22.177.20", port 51599
2026-08-11 10:08:09.449 UTC [2545] LOG: listening on Unix socket "/tmp/.s.PGSQL.51599"
2026-08-11 10:08:09.474 UTC [2548] LOG: database system was shut down at 2026-08-11 10:08:07 UTC
2026-08-11 10:08:09.479 UTC [2545] LOG: database system is ready to accept connections
I20260811 10:08:10.514050 23236 mini_postgres.cc:96] Postgres bound to 51599
2026-08-11 10:08:10.522 UTC [2556] FATAL: database "slave" does not exist
127.22.177.20:51599 - accepting connections
I20260811 10:08:10.523607 23236 mini_ranger.cc:161] Starting Ranger...
I20260811 10:08:10.542969 23236 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 10:08:11.216426 23236 mini_ranger.cc:87] Created ranger Postgres database
I20260811 10:08:11.216533 23236 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-admin
2026-08-11 10:08:11,552 [I] DB FLAVOR :POSTGRES
2026-08-11 10:08:11,552 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:08:11,553 [I] Checking connection..
2026-08-11 10:08:11,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 10:08:11,858 [I] Checking connection passed.
2026-08-11 10:08:11,858 [I] --------- Verifying version history table ---------
2026-08-11 10:08:11,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:08:12,143 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:08:12,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:08:12,420 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:08:12,421 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 10:08:12,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 10:08:12.703 UTC [2623] WARNING: there is no transaction in progress
2026-08-11 10:08:12,711 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 10:08:12,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:08:13,000 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 10:08:13,001 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:08:13,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 10:08:13,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 10:08:13,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-g350.c.gcp-upstream.internal','N') ;"
2026-08-11 10:08:13,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:08:14,075 [I] Table x_portal_user does not exist in database ranger
2026-08-11 10:08:14,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:08:14,344 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 10:08:14,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:08:14,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:08:14,895 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 10:08:14,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 10:08:16.185 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.208 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.231 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.253 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.277 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.303 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.375 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.385 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.403 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.417 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.431 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.449 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.462 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.472 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.482 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:16.493 UTC [2746] WARNING: there is no transaction in progress
2026-08-11 10:08:17,452 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 10:08:17,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 10:08:17,738 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 10:08:17,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:08:18,034 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 10:08:18,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:08:18,335 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 10:08:18,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:08:18,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:08:18,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-g350.c.gcp-upstream.internal';"
2026-08-11 10:08:19,147 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 10:08:19,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 10:08:19,408 [I] DB_PATCHES have already been applied
I20260811 10:08:19.414404 23236 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-admin:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2wzoa9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 10:08:19.414477 23236 mini_ranger.cc:193] Using host: 127.22.177.20
I20260811 10:08:19.418725 23236 mini_ranger.cc:227] Ranger admin URL: http://127.22.177.20:41273
Aug 11, 2026 10:08:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:08:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:08:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:08:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:08:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 10:08:20.644284 23236 mini_ranger.cc:160] Time spent starting Ranger: real 10.121s user 0.001s sys 0.012s
Aug 11, 2026 10:08:41 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:08:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 10:08:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:08:42 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:08:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 10:08:43.658398 23236 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/rangerkms_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/rangerkms_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.177.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.177.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 10:08:43 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442923, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 10:08:43.773010 23236 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 10:08:43.785290 23236 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 10:08:45.294966 23236 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 10:08:45.295051 23236 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms
I20260811 10:08:45.295064 23236 mini_ranger_kms.cc:227] Using postgres at 127.22.177.20:51599
2026-08-11 10:08:45,598 [I] DB FLAVOR :POSTGRES
2026-08-11 10:08:45,598 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:08:45,598 [I] Checking connection
2026-08-11 10:08:45,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 10:08:45,893 [I] connection success
2026-08-11 10:08:45,894 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 10:08:45,894 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 10:08:45,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 10:08:46,169 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 10:08:46,169 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:08:46,170 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 10:08:46,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:51599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 10:08:46,464 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 10:08:47.033042 23236 mini_ranger_kms.cc:310] Created kms service
I20260811 10:08:47.278134 23236 mini_ranger_kms.cc:326] Created kudu user
I20260811 10:08:47.330377 23236 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 10:08:47 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442923, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:08:47.760872 23236 mini_ranger_kms.cc:384] Added ranger policy
I20260811 10:08:47.761031 23236 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms:/tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 10:08:47.761055 23236 mini_ranger_kms.cc:240] Using host: 127.22.177.20
I20260811 10:08:47.766289 23236 mini_ranger_kms.cc:276] Ranger KMS PID: 2944
I20260811 10:08:47.766438 23236 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.177.20:53167
Aug 11, 2026 10:08:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:08:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:08:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:08:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:08:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:08:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 10:08:49 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786442929, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.177.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:50.261539 23236 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.489s user 0.010s sys 0.267s
I20260811 10:08:50.261686 23236 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 10:08:50.261772 23236 mini_ranger_kms.cc:401] 127.22.177.20:53167/kms/v1/keys
Aug 11 10:08:50 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442929, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.177.20@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
Aug 11 10:08:50 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442929, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.177.20@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
Aug 11 10:08:53 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:08:53.385870 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42023
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:42023
--ranger_config_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:53167/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:53.475932 3006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:53.476168 3006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:53.476191 3006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:53.477530 3006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:08:53.477614 3006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:53.477630 3006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:53.477643 3006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:08:53.477655 3006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:08:53.479604 3006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.177.20:53167/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:42023
--ranger_config_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:53.479969 3006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:53.480250 3006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:53.486418 3012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:53.486892 3011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:53.486907 3014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:53.487056 3006 server_base.cc:1034] running on GCE node
I20260811 10:08:53.487458 3006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:53.487810 3006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:53.488996 3006 hybrid_clock.cc:648] HybridClock initialized: now 1786442933488950 us; error 48 us; skew 500 ppm
Aug 11 10:08:53 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442933, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:53.491895 3006 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:08:53.492467 3006 webserver.cc:460] Webserver started at http://127.22.177.62:39447/ using document root <none> and password file <none>
I20260811 10:08:53.492725 3006 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:53.492792 3006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:53.492952 3006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:08:53 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442933, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:08:53.562943 3006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "e14e8bb134104c5faad1632a129e1b4d"
format_stamp: "Formatted at 2026-08-11 10:08:53 on dist-test-slave-g350"
server_key: "ec7014bf7b2550fd4e7e4d1caad2ece4"
server_key_iv: "fd4310afdb26fb86d33e9a3a69ed6acf"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:53.563529 3006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "e14e8bb134104c5faad1632a129e1b4d"
format_stamp: "Formatted at 2026-08-11 10:08:53 on dist-test-slave-g350"
server_key: "ec7014bf7b2550fd4e7e4d1caad2ece4"
server_key_iv: "fd4310afdb26fb86d33e9a3a69ed6acf"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:53.565263 3006 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:08:53.581112 3023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:53.581314 3006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:08:53.581413 3006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e14e8bb134104c5faad1632a129e1b4d"
format_stamp: "Formatted at 2026-08-11 10:08:53 on dist-test-slave-g350"
server_key: "ec7014bf7b2550fd4e7e4d1caad2ece4"
server_key_iv: "fd4310afdb26fb86d33e9a3a69ed6acf"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:53.581516 3006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:53.596837 3006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:53.602456 3006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:53.602864 3006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:53.614187 3006 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:42023
I20260811 10:08:53.615355 3006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:08:53.615402 3081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:42023 every 8 connection(s)
I20260811 10:08:53.617913 3082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:53.623790 3082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d: Bootstrap starting.
I20260811 10:08:53.624857 3082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:53.625008 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 3006
I20260811 10:08:53.625207 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:08:53.625484 3082 log.cc:824] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:53.625710 23236 external_mini_cluster.cc:1428] Setting key c65a3e95510f7ad76454673680f8c6ce
I20260811 10:08:53.627282 3082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d: No bootstrap required, opened a new log
I20260811 10:08:53.633030 3082 raft_consensus.cc:348] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e14e8bb134104c5faad1632a129e1b4d" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42023 } }
I20260811 10:08:53.633420 3082 raft_consensus.cc:374] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:53.633517 3082 raft_consensus.cc:729] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e14e8bb134104c5faad1632a129e1b4d, State: Initialized, Role: FOLLOWER
I20260811 10:08:53.633865 3082 consensus_queue.cc:260] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e14e8bb134104c5faad1632a129e1b4d" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42023 } }
I20260811 10:08:53.634016 3082 raft_consensus.cc:388] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:08:53.634069 3082 raft_consensus.cc:482] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:08:53.634204 3082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:53.639310 3082 raft_consensus.cc:504] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e14e8bb134104c5faad1632a129e1b4d" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42023 } }
I20260811 10:08:53.639828 3082 leader_election.cc:302] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e14e8bb134104c5faad1632a129e1b4d; no voters:
I20260811 10:08:53.640254 3082 leader_election.cc:288] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:08:53.640424 3087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:08:53.640863 3087 raft_consensus.cc:686] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [term 1 LEADER]: Becoming Leader. State: Replica: e14e8bb134104c5faad1632a129e1b4d, State: Running, Role: LEADER
I20260811 10:08:53.640900 3082 sys_catalog.cc:564] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:08:53.641171 3087 consensus_queue.cc:237] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e14e8bb134104c5faad1632a129e1b4d" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42023 } }
I20260811 10:08:53.641932 3087 sys_catalog.cc:455] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e14e8bb134104c5faad1632a129e1b4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e14e8bb134104c5faad1632a129e1b4d" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42023 } } }
Aug 11 10:08:53 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:53.642132 3087 sys_catalog.cc:458] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:53.642367 3087 sys_catalog.cc:455] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e14e8bb134104c5faad1632a129e1b4d. Latest consensus state: current_term: 1 leader_uuid: "e14e8bb134104c5faad1632a129e1b4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e14e8bb134104c5faad1632a129e1b4d" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42023 } } }
I20260811 10:08:53.642589 3087 sys_catalog.cc:458] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d [sys.catalog]: This master's current role is: LEADER
I20260811 10:08:53.642926 3099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:08:53.645224 3099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:08:53.646251 3082 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 10:08:53.649464 3107 mvcc.cc:205] Tried to move back new op lower bound from 7317270256221536256 to 7317270256195342336. Current Snapshot: MvccSnapshot[applied={T|T < 7317270256221536256}]
I20260811 10:08:53.650866 3099 catalog_manager.cc:1347] Generated new cluster ID: acb1b755235c4a4db143d9f7d7637327
I20260811 10:08:53.651036 3099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:08:53.652591 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:53.626657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.22.177.62:42023)
0811 10:08:53.626965 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:53.626991 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:53.627213 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:08:53.628301 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:53.628310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:53.628398 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:53.628733 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:53.630032 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:53.631743 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:53.633004 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:53.633987 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:53.634214 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:53.643386 (+ 9172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:53.643402 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:53.643403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:53.643419 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:53.647951 (+ 4532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:53.648982 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:53.648995 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:53.648996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:53.649251 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:53.649903 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:53.649905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:53.649905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:53.650494 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:53.650822 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:53.651873 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0811 10:08:53.652169 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":112,"threads_started":1}
I20260811 10:08:53.690220 3099 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:08:53.691730 3099 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:08:53.700726 3099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d: Generated new TSK 0
I20260811 10:08:53.701258 3099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:08:54.681870 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:53167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42023
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:54.811462 3132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:54.811764 3132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:54.811805 3132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:54.813656 3132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:54.813725 3132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:54.813771 3132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:08:54.815647 3132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.177.20:53167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:54.815963 3132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:54.816244 3132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:54.822391 3138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:54.822499 3139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:54.822540 3132 server_base.cc:1034] running on GCE node
W20260811 10:08:54.822384 3141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:54.822921 3132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:54.823246 3132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:54.824486 3132 hybrid_clock.cc:648] HybridClock initialized: now 1786442934824459 us; error 52 us; skew 500 ppm
Aug 11 10:08:54 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442934, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:54.828013 3132 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:08:54.828678 3132 webserver.cc:460] Webserver started at http://127.22.177.1:38851/ using document root <none> and password file <none>
I20260811 10:08:54.829002 3132 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:54.829092 3132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:54.829310 3132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:08:54 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442934, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:08:54.846822 3132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "57a588b2aeed4af5b602d4c2d424cdfd"
format_stamp: "Formatted at 2026-08-11 10:08:54 on dist-test-slave-g350"
server_key: "55e0e30060c083f2726fb38591a02f3b"
server_key_iv: "4f4962497c3d0fb0fc0065585c693aed"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:54.847568 3132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "57a588b2aeed4af5b602d4c2d424cdfd"
format_stamp: "Formatted at 2026-08-11 10:08:54 on dist-test-slave-g350"
server_key: "55e0e30060c083f2726fb38591a02f3b"
server_key_iv: "4f4962497c3d0fb0fc0065585c693aed"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:54.850023 3132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 10:08:54.867208 3154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:54.867516 3132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 10:08:54.867635 3132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "57a588b2aeed4af5b602d4c2d424cdfd"
format_stamp: "Formatted at 2026-08-11 10:08:54 on dist-test-slave-g350"
server_key: "55e0e30060c083f2726fb38591a02f3b"
server_key_iv: "4f4962497c3d0fb0fc0065585c693aed"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:54.867754 3132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:54.890769 3132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:54.893895 3132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:54.894173 3132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:54.894536 3132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:54.895045 3132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:54.895153 3132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:54.895267 3132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:54.895366 3132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:54.903259 3132 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:33243
I20260811 10:08:54.903779 3132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:08:54.907545 3279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:33243 every 8 connection(s)
I20260811 10:08:54.913882 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 3132
I20260811 10:08:54.914026 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:08:54.914235 23236 external_mini_cluster.cc:1428] Setting key 7fcac92a4aeaa9d8584599afbb8a0511
Aug 11 10:08:54 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442934, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:54.924398 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:54.908680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:54039 (local address 127.22.177.62:42023)
0811 10:08:54.909706 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:54.909708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:54.909720 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:08:54.909749 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:54.909750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:54.909764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:54.909844 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:54.910718 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:54.911720 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:54.913441 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:54.913816 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:54.913889 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:54.918227 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:54.918229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:54.918229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:54.918239 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:54.920388 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:54.921017 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:54.921019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:54.921019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:54.921101 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:54.923433 (+ 2332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:54.923434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:54.923435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:54.923613 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:54.923686 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:54.924153 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 10:08:54.924272 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":69,"threads_started":1}
I20260811 10:08:54.925038 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:54.908383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42023 (local address 127.22.177.1:54039)
0811 10:08:54.908632 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:54.908689 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:54.909421 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:54.909963 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:54.909968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:54.910261 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:54.910593 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:54.910600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:54.911853 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:54.911855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:54.913316 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:54.913320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:54.913978 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:54.913979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:54.914170 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:54.916115 (+ 1945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:54.916129 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:54.918076 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:54.920514 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:54.920525 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:54.920528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:54.920911 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:54.923097 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:54.923099 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:54.923099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:54.923315 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:54.923732 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:54.923741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:54.923958 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 10:08:54.924529 (+ 571us) client_negotiation.cc:240] Negotiation successful
0811 10:08:54.924739 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":119,"threads_started":1}
I20260811 10:08:54.925882 3280 heartbeater.cc:347] Connected to a master server at 127.22.177.62:42023
I20260811 10:08:54.925999 3280 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:54.926213 3280 heartbeater.cc:511] Master 127.22.177.62:42023 requested a full tablet report, sending...
I20260811 10:08:54.926899 3042 ts_manager.cc:194] Registered new tserver with Master: 57a588b2aeed4af5b602d4c2d424cdfd (127.22.177.1:33243)
I20260811 10:08:54.927946 3042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:54039
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:08:54.960515 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:53167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42023
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 10:08:55.096984 3286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:55.097316 3286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:55.097355 3286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:55.099342 3286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:55.099422 3286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:55.099467 3286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:08:55.102010 3286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.177.20:53167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:55.102309 3286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:55.102669 3286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:55.111109 3294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:55.111129 3297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:55.111645 3293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:55.112064 3286 server_base.cc:1034] running on GCE node
I20260811 10:08:55.112317 3286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:55.112629 3286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:55.113890 3286 hybrid_clock.cc:648] HybridClock initialized: now 1786442935113853 us; error 52 us; skew 500 ppm
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:55.117576 3286 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:08:55.118286 3286 webserver.cc:460] Webserver started at http://127.22.177.2:46185/ using document root <none> and password file <none>
I20260811 10:08:55.118558 3286 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:55.118606 3286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:55.118709 3286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:08:55.138990 3286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "b82a591972ea42e695ea624f29ffd20b"
format_stamp: "Formatted at 2026-08-11 10:08:55 on dist-test-slave-g350"
server_key: "7f5bcc6057c638b9676286c28b3ca446"
server_key_iv: "039430ab88d9eb760bc6cd77babcf923"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:55.140043 3286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "b82a591972ea42e695ea624f29ffd20b"
format_stamp: "Formatted at 2026-08-11 10:08:55 on dist-test-slave-g350"
server_key: "7f5bcc6057c638b9676286c28b3ca446"
server_key_iv: "039430ab88d9eb760bc6cd77babcf923"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:55.142922 3286 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:08:55.164901 3306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:55.165155 3286 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 10:08:55.165292 3286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "b82a591972ea42e695ea624f29ffd20b"
format_stamp: "Formatted at 2026-08-11 10:08:55 on dist-test-slave-g350"
server_key: "7f5bcc6057c638b9676286c28b3ca446"
server_key_iv: "039430ab88d9eb760bc6cd77babcf923"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:55.165401 3286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:55.184392 3286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:55.187170 3286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:55.187354 3286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:55.187671 3286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:55.188153 3286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:55.188215 3286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:55.188248 3286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:55.188266 3286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:55.197207 3286 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:38687
I20260811 10:08:55.197301 3431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:38687 every 8 connection(s)
I20260811 10:08:55.197861 3286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:08:55.200306 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 3286
I20260811 10:08:55.200431 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:08:55.200649 23236 external_mini_cluster.cc:1428] Setting key 5571e64a7dec12934d48ace8a1168e6c
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:55.212121 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.198736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:37895 (local address 127.22.177.62:42023)
0811 10:08:55.198793 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.198795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.199113 (+ 318us) server_negotiation.cc:407] Connection header received
0811 10:08:55.199852 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.199854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.199876 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.199941 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:55.200878 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.201703 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.203257 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.203564 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.203621 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.206729 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:55.206732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:55.206733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:55.206742 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:55.210104 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:55.210802 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:55.210804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:55.210804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:55.210876 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:55.211215 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:55.211217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:55.211217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:55.211379 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:55.211429 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.211877 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.211985 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:08:55.213016 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.198712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42023 (local address 127.22.177.2:37895)
0811 10:08:55.198937 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.198992 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.199715 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.200085 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.200090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.200371 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:55.200726 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.200734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.201850 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.201851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.203137 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.203140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.203726 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.203727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.203932 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.204650 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:55.204663 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:55.206572 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:55.210245 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:55.210260 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:55.210264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:55.210678 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:55.210965 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:55.210966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:55.210966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:55.211124 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:55.211484 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:55.211491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:55.211678 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.212542 (+ 864us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.212738 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260811 10:08:55.213846 3432 heartbeater.cc:347] Connected to a master server at 127.22.177.62:42023
I20260811 10:08:55.213954 3432 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:55.214165 3432 heartbeater.cc:511] Master 127.22.177.62:42023 requested a full tablet report, sending...
I20260811 10:08:55.214643 3042 ts_manager.cc:194] Registered new tserver with Master: b82a591972ea42e695ea624f29ffd20b (127.22.177.2:38687)
I20260811 10:08:55.215163 3042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:37895
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:08:55.251844 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:53167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42023
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:08:55.368809 3437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:08:55.369158 3437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:08:55.369199 3437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:08:55.371122 3437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:08:55.371209 3437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:08:55.371254 3437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:08:55.373752 3437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.177.20:53167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:42023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:08:55.374058 3437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:08:55.374352 3437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:08:55.381083 3446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:55.381407 3443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:08:55.381103 3444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:08:55.381220 3437 server_base.cc:1034] running on GCE node
I20260811 10:08:55.382009 3437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:08:55.382413 3437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:08:55.383625 3437 hybrid_clock.cc:648] HybridClock initialized: now 1786442935383593 us; error 67 us; skew 500 ppm
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:08:55.395584 3437 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:08:55.396258 3437 webserver.cc:460] Webserver started at http://127.22.177.3:40499/ using document root <none> and password file <none>
I20260811 10:08:55.396600 3437 fs_manager.cc:359] Metadata directory not provided
I20260811 10:08:55.396690 3437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:08:55.396927 3437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:08:55.412966 3437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556"
format_stamp: "Formatted at 2026-08-11 10:08:55 on dist-test-slave-g350"
server_key: "240b95260979f9b590b7237ad91599c1"
server_key_iv: "276d640723a299163ca47292cbdbb84d"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:55.413615 3437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556"
format_stamp: "Formatted at 2026-08-11 10:08:55 on dist-test-slave-g350"
server_key: "240b95260979f9b590b7237ad91599c1"
server_key_iv: "276d640723a299163ca47292cbdbb84d"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:55.417007 3437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 10:08:55.437837 3453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:55.438108 3437 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.002s
I20260811 10:08:55.438216 3437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556"
format_stamp: "Formatted at 2026-08-11 10:08:55 on dist-test-slave-g350"
server_key: "240b95260979f9b590b7237ad91599c1"
server_key_iv: "276d640723a299163ca47292cbdbb84d"
server_key_version: "kuduclusterkey@0"
I20260811 10:08:55.438325 3437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:08:55.459215 3437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:08:55.462016 3437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:08:55.462256 3437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:08:55.462533 3437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:08:55.462996 3437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:08:55.463079 3437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:55.463143 3437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:08:55.463193 3437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:08:55.472249 3437 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:44861
I20260811 10:08:55.472582 3577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:44861 every 8 connection(s)
I20260811 10:08:55.472807 3437 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:08:55.476434 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 3437
I20260811 10:08:55.476560 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:08:55.476755 23236 external_mini_cluster.cc:1428] Setting key 0e21bf0c2353d39fba9d0950f33fb3eb
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:08:55.486152 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.473975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:34221 (local address 127.22.177.62:42023)
0811 10:08:55.474046 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.474049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.474353 (+ 304us) server_negotiation.cc:407] Connection header received
0811 10:08:55.475154 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.475156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.475169 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.475203 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:55.476267 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.477040 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.478635 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.478936 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.478986 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.482126 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:55.482129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:55.482129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:55.482138 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:55.484111 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:55.484712 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:55.484713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:55.484714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:55.484792 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:55.485165 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:55.485166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:55.485167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:55.485394 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:55.485505 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.485949 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.486046 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:08:55.487108 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.473655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42023 (local address 127.22.177.3:34221)
0811 10:08:55.474194 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.474249 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.475009 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.475418 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.475423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.475756 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:08:55.476137 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.476144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.477170 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.477171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.478522 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.478525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.479090 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.479091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.479241 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.479957 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:08:55.479968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:08:55.481989 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:08:55.484224 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:55.484234 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:55.484237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:55.484594 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:55.484868 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:08:55.484869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:08:55.484870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:08:55.485025 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:08:55.485503 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:08:55.485511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:08:55.485757 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.486639 (+ 882us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.486853 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":89,"threads_started":1}
I20260811 10:08:55.487653 3578 heartbeater.cc:347] Connected to a master server at 127.22.177.62:42023
I20260811 10:08:55.487778 3578 heartbeater.cc:464] Registering TS with master...
I20260811 10:08:55.488030 3578 heartbeater.cc:511] Master 127.22.177.62:42023 requested a full tablet report, sending...
I20260811 10:08:55.488469 3042 ts_manager.cc:194] Registered new tserver with Master: c3a20d88a4d94f27a5a9e4c0cf0d2556 (127.22.177.3:44861)
I20260811 10:08:55.488705 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:08:55.488998 3042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:34221
I20260811 10:08:55.499599 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.490644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.22.177.62:42023)
0811 10:08:55.490699 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.490702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.490871 (+ 169us) server_negotiation.cc:407] Connection header received
0811 10:08:55.490926 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.490927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.490941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.491011 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:08:55.491906 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.492654 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.493918 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.494230 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.494284 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.496217 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:08:55.496219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:08:55.496219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:08:55.496228 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:08:55.498185 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:55.498660 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:55.498661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:55.498661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:55.498724 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:08:55.498951 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:08:55.498952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:08:55.498952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:08:55.499096 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:08:55.499173 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.499406 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.499498 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:08:55.502604 3042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:08:55.505133 3042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:08:55.516245 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.510363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:38687 (local address 127.0.0.1:39372)
0811 10:08:55.511446 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.511456 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.511506 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.511853 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.511857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.511859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:55.511936 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.511939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.513340 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.513341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.514729 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.514732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.515891 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.515893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.515980 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.515996 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.516073 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.516150 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":119,"threads_started":1}
I20260811 10:08:55.517014 3593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.510475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.22.177.2:38687)
0811 10:08:55.511390 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.511394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.511556 (+ 162us) server_negotiation.cc:407] Connection header received
0811 10:08:55.511601 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.511604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.511620 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.511753 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:55.512034 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.513238 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.514854 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.515663 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.515739 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.515830 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.516878 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.516926 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":88,"threads_started":1}
I20260811 10:08:55.517946 3359 tablet_service.cc:1467] Processing CreateTablet for tablet 25c36a4b37414e44acf997765d6c0a03 (DEFAULT_TABLE table=test-table [id=e1f094efc3ca4d689239a3000eaa6b98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:08:55.518348 3359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c36a4b37414e44acf997765d6c0a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:55.521437 3592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.510356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:44861 (local address 127.0.0.1:38368)
0811 10:08:55.511070 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.511125 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.511216 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.512825 (+ 1609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.512827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.512829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:55.512898 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.512900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.518041 (+ 5141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.518042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.519278 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.519281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.521138 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.521140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.521229 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.521234 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.521301 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.521350 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":166,"threads_started":1}
I20260811 10:08:55.521744 3595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.510703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.22.177.3:44861)
0811 10:08:55.510993 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.511000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.511227 (+ 227us) server_negotiation.cc:407] Connection header received
0811 10:08:55.511670 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.511673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.511693 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.512723 (+ 1030us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:55.512988 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.517930 (+ 4942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.519417 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.520940 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.521017 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.521098 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.521574 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.521661 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":125,"threads_started":1}
I20260811 10:08:55.522680 3502 tablet_service.cc:1467] Processing CreateTablet for tablet 25c36a4b37414e44acf997765d6c0a03 (DEFAULT_TABLE table=test-table [id=e1f094efc3ca4d689239a3000eaa6b98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:08:55.523087 3502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c36a4b37414e44acf997765d6c0a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:55.523286 3597 tablet_bootstrap.cc:492] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Bootstrap starting.
I20260811 10:08:55.524587 3597 tablet_bootstrap.cc:654] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:55.524948 3597 log.cc:824] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:55.525820 3597 tablet_bootstrap.cc:492] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: No bootstrap required, opened a new log
I20260811 10:08:55.525887 3597 ts_tablet_manager.cc:1397] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:08:55.528064 3597 raft_consensus.cc:348] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.528258 3597 raft_consensus.cc:374] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:55.528292 3597 raft_consensus.cc:729] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b82a591972ea42e695ea624f29ffd20b, State: Initialized, Role: FOLLOWER
I20260811 10:08:55.528410 3597 consensus_queue.cc:260] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.528716 3597 ts_tablet_manager.cc:1428] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:08:55.528843 3432 heartbeater.cc:503] Master 127.22.177.62:42023 was elected leader, sending a full tablet report...
I20260811 10:08:55.529474 3599 tablet_bootstrap.cc:492] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Bootstrap starting.
I20260811 10:08:55.530745 3599 tablet_bootstrap.cc:654] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:55.530779 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.510644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:33243 (local address 127.0.0.1:60142)
0811 10:08:55.516750 (+ 6106us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.516762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.516797 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.523098 (+ 6301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.523099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.523102 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:55.523142 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.523155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.525198 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.525200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.529313 (+ 4113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.529317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.530496 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.530497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.530573 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.530578 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.530642 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.530698 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6092,"thread_start_us":64,"threads_started":1}
I20260811 10:08:55.531020 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.510844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.22.177.1:33243)
0811 10:08:55.522915 (+ 12071us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.522921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.522932 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:08:55.522969 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.522971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.522989 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.523088 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:55.523264 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.525070 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.529455 (+ 4385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.530298 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.530392 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.530456 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.530903 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.530953 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12028,"thread_start_us":71,"threads_started":1}
I20260811 10:08:55.531457 3599 log.cc:824] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:55.531811 3206 tablet_service.cc:1467] Processing CreateTablet for tablet 25c36a4b37414e44acf997765d6c0a03 (DEFAULT_TABLE table=test-table [id=e1f094efc3ca4d689239a3000eaa6b98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:08:55.532219 3206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c36a4b37414e44acf997765d6c0a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:08:55.532434 3599 tablet_bootstrap.cc:492] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: No bootstrap required, opened a new log
I20260811 10:08:55.532498 3599 ts_tablet_manager.cc:1397] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:08:55.534855 3599 raft_consensus.cc:348] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.535032 3599 raft_consensus.cc:374] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:55.535064 3599 raft_consensus.cc:729] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a20d88a4d94f27a5a9e4c0cf0d2556, State: Initialized, Role: FOLLOWER
I20260811 10:08:55.535223 3599 consensus_queue.cc:260] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.535513 3599 ts_tablet_manager.cc:1428] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:08:55.535782 3578 heartbeater.cc:503] Master 127.22.177.62:42023 was elected leader, sending a full tablet report...
I20260811 10:08:55.536438 3603 tablet_bootstrap.cc:492] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd: Bootstrap starting.
I20260811 10:08:55.537485 3603 tablet_bootstrap.cc:654] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd: Neither blocks nor log segments found. Creating new log.
I20260811 10:08:55.537974 3603 log.cc:824] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:08:55.538934 3602 raft_consensus.cc:482] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:08:55.539067 3602 raft_consensus.cc:504] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.539451 3602 leader_election.cc:288] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b82a591972ea42e695ea624f29ffd20b (127.22.177.2:38687), 57a588b2aeed4af5b602d4c2d424cdfd (127.22.177.1:33243)
I20260811 10:08:55.540337 3603 tablet_bootstrap.cc:492] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd: No bootstrap required, opened a new log
I20260811 10:08:55.540537 3603 ts_tablet_manager.cc:1397] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 10:08:55.544032 3603 raft_consensus.cc:348] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.544366 3603 raft_consensus.cc:374] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:08:55.544463 3603 raft_consensus.cc:729] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57a588b2aeed4af5b602d4c2d424cdfd, State: Initialized, Role: FOLLOWER
I20260811 10:08:55.544704 3603 consensus_queue.cc:260] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.545145 3603 ts_tablet_manager.cc:1428] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 10:08:55.545539 3280 heartbeater.cc:503] Master 127.22.177.62:42023 was elected leader, sending a full tablet report...
I20260811 10:08:55.547122 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.539621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:38687 (local address 127.22.177.3:56723)
0811 10:08:55.539667 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.539682 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.539746 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.540125 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.540127 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.540129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:55.540205 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.540207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.541230 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.541232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.542437 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.542440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.546834 (+ 4394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.546835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.546921 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.546925 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.546985 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.547037 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:08:55.547956 3593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.539932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:56723 (local address 127.22.177.2:38687)
0811 10:08:55.539984 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.539987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.539996 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:08:55.540027 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.540028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.540038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.540108 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:55.540305 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.541085 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.545970 (+ 4885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.546689 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.546779 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.546790 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.547789 (+ 999us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.547884 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:08:55.548307 3382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c36a4b37414e44acf997765d6c0a03" candidate_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b82a591972ea42e695ea624f29ffd20b" is_pre_election: true
I20260811 10:08:55.548478 3382 raft_consensus.cc:2454] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3a20d88a4d94f27a5a9e4c0cf0d2556 in term 0.
I20260811 10:08:55.549084 3454 leader_election.cc:302] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b82a591972ea42e695ea624f29ffd20b, c3a20d88a4d94f27a5a9e4c0cf0d2556; no voters:
I20260811 10:08:55.549489 3602 raft_consensus.cc:2781] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:08:55.549774 3602 raft_consensus.cc:482] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:08:55.549820 3602 raft_consensus.cc:3037] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:55.551002 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.539959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:35399 (local address 127.22.177.1:33243)
0811 10:08:55.540000 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.540002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.545417 (+ 5415us) server_negotiation.cc:407] Connection header received
0811 10:08:55.546610 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.546611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.546621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.546651 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:08:55.546934 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.547740 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.549769 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.550439 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.550499 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.550509 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.550805 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.550867 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:08:55.551369 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.539714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:33243 (local address 127.22.177.3:35399)
0811 10:08:55.545311 (+ 5597us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.545328 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:08:55.546495 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:08:55.546718 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:08:55.546719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:08:55.546721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:08:55.546843 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.546846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.548072 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.548074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.549644 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:08:55.549649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.550605 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.550607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:08:55.550685 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.550690 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:08:55.551164 (+ 474us) client_negotiation.cc:240] Negotiation successful
0811 10:08:55.551235 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5563,"thread_start_us":90,"threads_started":1}
I20260811 10:08:55.551550 3602 raft_consensus.cc:504] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.551734 3602 leader_election.cc:288] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [CANDIDATE]: Term 1 election: Requested vote from peers b82a591972ea42e695ea624f29ffd20b (127.22.177.2:38687), 57a588b2aeed4af5b602d4c2d424cdfd (127.22.177.1:33243)
I20260811 10:08:55.551769 3228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c36a4b37414e44acf997765d6c0a03" candidate_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" is_pre_election: true
I20260811 10:08:55.551956 3228 raft_consensus.cc:2454] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3a20d88a4d94f27a5a9e4c0cf0d2556 in term 0.
I20260811 10:08:55.552717 3228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c36a4b37414e44acf997765d6c0a03" candidate_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57a588b2aeed4af5b602d4c2d424cdfd"
I20260811 10:08:55.552809 3228 raft_consensus.cc:3037] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:55.555686 3228 raft_consensus.cc:2454] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3a20d88a4d94f27a5a9e4c0cf0d2556 in term 1.
I20260811 10:08:55.556073 3382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c36a4b37414e44acf997765d6c0a03" candidate_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b82a591972ea42e695ea624f29ffd20b"
I20260811 10:08:55.556203 3382 raft_consensus.cc:3037] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:08:55.557636 3382 raft_consensus.cc:2454] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3a20d88a4d94f27a5a9e4c0cf0d2556 in term 1.
I20260811 10:08:55.558235 3454 leader_election.cc:302] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57a588b2aeed4af5b602d4c2d424cdfd, c3a20d88a4d94f27a5a9e4c0cf0d2556; no voters:
I20260811 10:08:55.558413 3602 raft_consensus.cc:2781] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:08:55.558743 3602 raft_consensus.cc:686] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 1 LEADER]: Becoming Leader. State: Replica: c3a20d88a4d94f27a5a9e4c0cf0d2556, State: Running, Role: LEADER
I20260811 10:08:55.559173 3602 consensus_queue.cc:237] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } }
I20260811 10:08:55.561112 3042 catalog_manager.cc:5495] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3a20d88a4d94f27a5a9e4c0cf0d2556 (127.22.177.3). New cstate: current_term: 1 leader_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3a20d88a4d94f27a5a9e4c0cf0d2556" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 44861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 } health_report { overall_health: UNKNOWN } } }
I20260811 10:08:55.580619 3595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.575930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.22.177.3:44861)
0811 10:08:55.575981 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:08:55.575984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:08:55.575992 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:08:55.576042 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:08:55.576042 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:08:55.576067 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:08:55.576146 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:08:55.576396 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.577333 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.579015 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:08:55.579316 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:08:55.579390 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:08:55.579618 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:08:55.579768 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:08:55.579861 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 10:08:55.580140 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 10:08:55.580170 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:08:55.584077 3228 raft_consensus.cc:1261] T 25c36a4b37414e44acf997765d6c0a03 P 57a588b2aeed4af5b602d4c2d424cdfd [term 1 FOLLOWER]: Refusing update from remote peer c3a20d88a4d94f27a5a9e4c0cf0d2556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:08:55.584604 3602 consensus_queue.cc:1035] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57a588b2aeed4af5b602d4c2d424cdfd" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:08:55.587728 3382 raft_consensus.cc:1261] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 1 FOLLOWER]: Refusing update from remote peer c3a20d88a4d94f27a5a9e4c0cf0d2556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:08:55.588155 3602 consensus_queue.cc:1035] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b82a591972ea42e695ea624f29ffd20b" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 11 10:08:55 dist-test-slave-g350 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442935, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:08:55.593371 3611 mvcc.cc:205] Tried to move back new op lower bound from 7317270264148840448 to 7317270264051957760. Current Snapshot: MvccSnapshot[applied={T|T < 7317270264148840448}]
I20260811 10:08:55.593837 3614 mvcc.cc:205] Tried to move back new op lower bound from 7317270264148840448 to 7317270264051957760. Current Snapshot: MvccSnapshot[applied={T|T < 7317270264148840448}]
I20260811 10:08:55.595712 3612 mvcc.cc:205] Tried to move back new op lower bound from 7317270264148840448 to 7317270264051957760. Current Snapshot: MvccSnapshot[applied={T|T < 7317270264148840448}]
I20260811 10:08:55.599503 3042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:08:55.599629 3042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:08:55.600585 3042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e14e8bb134104c5faad1632a129e1b4d: Sending DeleteTablet for 3 replicas of tablet 25c36a4b37414e44acf997765d6c0a03
I20260811 10:08:55.601336 3359 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c36a4b37414e44acf997765d6c0a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:08:55 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:39372
I20260811 10:08:55.601662 3622 tablet_replica.cc:330] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: stopping tablet replica
I20260811 10:08:55.601648 3206 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c36a4b37414e44acf997765d6c0a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:08:55 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:60142
I20260811 10:08:55.601894 3622 raft_consensus.cc:2229] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:08:55.601889 3502 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c36a4b37414e44acf997765d6c0a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:08:55 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:38368
I20260811 10:08:55.602084 3622 raft_consensus.cc:2258] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:08:55.602411 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 3132
I20260811 10:08:55.602681 3622 ts_tablet_manager.cc:1905] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:08:55.613054 3622 ts_tablet_manager.cc:1918] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:08:55.613406 3622 log.cc:1196] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/25c36a4b37414e44acf997765d6c0a03
I20260811 10:08:55.614254 3624 tablet_replica.cc:330] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: stopping tablet replica
I20260811 10:08:55.614331 3622 ts_tablet_manager.cc:1939] T 25c36a4b37414e44acf997765d6c0a03 P b82a591972ea42e695ea624f29ffd20b: Deleting consensus metadata
I20260811 10:08:55.614454 3624 raft_consensus.cc:2229] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:08:55.614647 3624 raft_consensus.cc:2258] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:08:55.614928 3024 catalog_manager.cc:4841] TS b82a591972ea42e695ea624f29ffd20b (127.22.177.2:38687): tablet 25c36a4b37414e44acf997765d6c0a03 (table test-table [id=e1f094efc3ca4d689239a3000eaa6b98]) successfully deleted
I20260811 10:08:55.623929 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 3286
I20260811 10:08:55.626171 3624 ts_tablet_manager.cc:1905] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 10:08:55.626637 3024 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:33243 (error 108)
I20260811 10:08:55.639330 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:08:55.639127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:33243 (local address 127.0.0.1:60156)
0811 10:08:55.639210 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 10:08:55.639272 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:33243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 10:08:55.639505 3024 catalog_manager.cc:4584] TS 57a588b2aeed4af5b602d4c2d424cdfd (127.22.177.1:33243): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25c36a4b37414e44acf997765d6c0a03: Network error: Client connection negotiation failed: client connection to 127.22.177.1:33243: connect: Connection refused (error 111)
I20260811 10:08:55.644065 3624 ts_tablet_manager.cc:1918] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:08:55.644209 3624 log.cc:1196] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/25c36a4b37414e44acf997765d6c0a03
I20260811 10:08:55.644644 3624 ts_tablet_manager.cc:1939] T 25c36a4b37414e44acf997765d6c0a03 P c3a20d88a4d94f27a5a9e4c0cf0d2556: Deleting consensus metadata
I20260811 10:08:55.645208 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 3437
I20260811 10:08:55.646385 3025 catalog_manager.cc:4841] TS c3a20d88a4d94f27a5a9e4c0cf0d2556 (127.22.177.3:44861): tablet 25c36a4b37414e44acf997765d6c0a03 (table test-table [id=e1f094efc3ca4d689239a3000eaa6b98]) successfully deleted
I20260811 10:08:55.654492 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 3006
I20260811 10:08:55.666110 23236 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 10:08:56.022185 23236 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 10:08:56.022322 23236 mini_ranger.cc:66] Stopping Ranger...
I20260811 10:08:56.369879 23236 mini_ranger.cc:68] Stopped Ranger
2026-08-11 10:08:56.370 UTC [2545] LOG: received smart shutdown request
2026-08-11 10:08:56.375 UTC [2545] LOG: background worker "logical replication launcher" (PID 2554) exited with exit code 1
2026-08-11 10:08:56.375 UTC [2549] LOG: shutting down
2026-08-11 10:08:56.636 UTC [2545] LOG: database system is shut down
2026-08-11T10:08:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52084 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:08:56 dist-test-slave-g350 krb5kdc[3634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:08:56 dist-test-slave-g350 krb5kdc[3634](info): set up 2 sockets
Aug 11 10:08:56 dist-test-slave-g350 krb5kdc[3634](info): commencing operation
krb5kdc: starting...
W20260811 10:08:57.892350 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:08:57 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:08:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/rangeradmin_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/rangeradmin_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/rangerlookup_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/rangerlookup_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
I20260811 10:08:58.097069 23236 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2wzoa9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/postgres -l logfile start
2026-08-11 10:09:01.171 UTC [3667] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 10:09:01.171 UTC [3667] LOG: listening on IPv4 address "127.22.177.20", port 44171
2026-08-11 10:09:01.176 UTC [3667] LOG: listening on Unix socket "/tmp/.s.PGSQL.44171"
2026-08-11 10:09:01.202 UTC [3670] LOG: database system was shut down at 2026-08-11 10:08:59 UTC
2026-08-11 10:09:01.207 UTC [3667] LOG: database system is ready to accept connections
I20260811 10:09:02.271479 23236 mini_postgres.cc:96] Postgres bound to 44171
2026-08-11 10:09:02.279 UTC [3678] FATAL: database "slave" does not exist
127.22.177.20:44171 - accepting connections
I20260811 10:09:02.279659 23236 mini_ranger.cc:161] Starting Ranger...
I20260811 10:09:02.296643 23236 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 10:09:03.002398 23236 mini_ranger.cc:87] Created ranger Postgres database
I20260811 10:09:03.002511 23236 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-admin
2026-08-11 10:09:03,295 [I] DB FLAVOR :POSTGRES
2026-08-11 10:09:03,296 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:09:03,296 [I] Checking connection..
2026-08-11 10:09:03,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 10:09:03,552 [I] Checking connection passed.
2026-08-11 10:09:03,552 [I] --------- Verifying version history table ---------
2026-08-11 10:09:03,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:09:03,838 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:09:03,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:09:04,134 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:09:04,135 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 10:09:04,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 10:09:04.393 UTC [3754] WARNING: there is no transaction in progress
2026-08-11 10:09:04,403 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 10:09:04,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:09:04,674 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 10:09:04,675 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:09:04,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 10:09:04,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 10:09:05,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-g350.c.gcp-upstream.internal','N') ;"
2026-08-11 10:09:05,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:09:05,838 [I] Table x_portal_user does not exist in database ranger
2026-08-11 10:09:05,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:09:06,108 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 10:09:06,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:09:06,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:09:06,667 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 10:09:06,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 10:09:07.916 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:07.938 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:07.963 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:07.984 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.004 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.027 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.108 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.121 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.143 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.159 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.173 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.197 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.211 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.223 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.240 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:08.258 UTC [3886] WARNING: there is no transaction in progress
2026-08-11 10:09:09,215 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 10:09:09,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 10:09:09,512 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 10:09:09,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:09:09,769 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 10:09:09,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:09:10,070 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 10:09:10,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:09:10,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:09:10,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-g350.c.gcp-upstream.internal';"
2026-08-11 10:09:10,882 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 10:09:10,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 10:09:11,187 [I] DB_PATCHES have already been applied
I20260811 10:09:11.193116 23236 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-admin:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2wzoa9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 10:09:11.193187 23236 mini_ranger.cc:193] Using host: 127.22.177.20
I20260811 10:09:11.197659 23236 mini_ranger.cc:227] Ranger admin URL: http://127.22.177.20:48253
Aug 11, 2026 10:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:09:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:09:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 10:09:12.432265 23236 mini_ranger.cc:160] Time spent starting Ranger: real 10.153s user 0.000s sys 0.012s
Aug 11, 2026 10:09:35 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:09:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:09:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:09:35 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:09:36 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 10:09:37.486697 23236 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.177.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.177.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/rangerkms_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.177.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/rangerkms_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.177.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.177.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.177.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/HTTP_127.22.177.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 10:09:37 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442977, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 10:09:37.611876 23236 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 10:09:37.627333 23236 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 10:09:38.676175 23236 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 10:09:38.676272 23236 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms
I20260811 10:09:38.676295 23236 mini_ranger_kms.cc:227] Using postgres at 127.22.177.20:44171
2026-08-11 10:09:38,864 [I] DB FLAVOR :POSTGRES
2026-08-11 10:09:38,865 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:09:38,865 [I] Checking connection
2026-08-11 10:09:38,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 10:09:39,165 [I] connection success
2026-08-11 10:09:39,165 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 10:09:39,166 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 10:09:39,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 10:09:39,435 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 10:09:39,435 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:09:39,435 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 10:09:39,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.177.20:44171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 10:09:39,746 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 10:09:40.349932 23236 mini_ranger_kms.cc:310] Created kms service
I20260811 10:09:40.570940 23236 mini_ranger_kms.cc:326] Created kudu user
I20260811 10:09:40.618448 23236 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 10:09:40 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442977, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:41.119552 23236 mini_ranger_kms.cc:384] Added ranger policy
I20260811 10:09:41.119679 23236 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms:/tmp/dist-test-task2wzoa9/build/release/bin/postgresql.jar:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2wzoa9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 10:09:41.119697 23236 mini_ranger_kms.cc:240] Using host: 127.22.177.20
I20260811 10:09:41.125092 23236 mini_ranger_kms.cc:276] Ranger KMS PID: 4102
I20260811 10:09:41.125233 23236 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.177.20:47933
Aug 11, 2026 10:09:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:09:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:09:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:09:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:09:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:09:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 10:09:43 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786442983, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.177.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:43.712339 23236 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.100s user 0.003s sys 0.167s
I20260811 10:09:43.712461 23236 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 10:09:43.712518 23236 mini_ranger_kms.cc:401] 127.22.177.20:47933/kms/v1/keys
Aug 11 10:09:44 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442983, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.177.20@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
Aug 11 10:09:44 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442983, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.177.20@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
Aug 11 10:09:47 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:09:47.161810 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40135
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40135
--ranger_config_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:47.255980 4154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:47.256258 4154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:47.256297 4154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:47.257531 4154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:47.257607 4154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:47.257678 4154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:47.257707 4154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:47.257736 4154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:47.259465 4154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.177.20:47933/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40135
--ranger_config_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:47.259779 4154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:47.260044 4154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:47.266285 4160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:47.266448 4161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:47.266285 4163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:47.266662 4154 server_base.cc:1034] running on GCE node
I20260811 10:09:47.266988 4154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:47.267315 4154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:47.268512 4154 hybrid_clock.cc:648] HybridClock initialized: now 1786442987268475 us; error 51 us; skew 500 ppm
Aug 11 10:09:47 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442987, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:47.271486 4154 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:09:47.272038 4154 webserver.cc:460] Webserver started at http://127.22.177.62:46351/ using document root <none> and password file <none>
I20260811 10:09:47.272298 4154 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:47.272360 4154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:47.272519 4154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:09:47 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442987, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:47.339062 4154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "924906c0df074b7285dec56100116f06"
format_stamp: "Formatted at 2026-08-11 10:09:47 on dist-test-slave-g350"
server_key: "157f1117e8d6d1197ff67f398f71c8b1"
server_key_iv: "e652b5d4710a564b12f06b2b05a989e7"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:47.339711 4154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "924906c0df074b7285dec56100116f06"
format_stamp: "Formatted at 2026-08-11 10:09:47 on dist-test-slave-g350"
server_key: "157f1117e8d6d1197ff67f398f71c8b1"
server_key_iv: "e652b5d4710a564b12f06b2b05a989e7"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:47.341686 4154 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:47.354414 4171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:47.354660 4154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:09:47.354773 4154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "924906c0df074b7285dec56100116f06"
format_stamp: "Formatted at 2026-08-11 10:09:47 on dist-test-slave-g350"
server_key: "157f1117e8d6d1197ff67f398f71c8b1"
server_key_iv: "e652b5d4710a564b12f06b2b05a989e7"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:47.354888 4154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:47.378671 4154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:47.381052 4154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:47.381258 4154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:47.386677 4154 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40135
I20260811 10:09:47.386705 4230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40135 every 8 connection(s)
I20260811 10:09:47.387117 4154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:09:47.387974 4231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:47.390260 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 4154
I20260811 10:09:47.390411 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:09:47.390628 23236 external_mini_cluster.cc:1428] Setting key 3f553b3dc2fcfb3355dc5513a55be29b
I20260811 10:09:47.391163 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Bootstrap starting.
I20260811 10:09:47.392059 4231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:47.392419 4231 log.cc:824] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:47.393288 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: No bootstrap required, opened a new log
I20260811 10:09:47.396027 4231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:47.396214 4231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:47.396302 4231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924906c0df074b7285dec56100116f06, State: Initialized, Role: FOLLOWER
I20260811 10:09:47.396463 4231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:47.396545 4231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:09:47.396577 4231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:09:47.396642 4231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:47.397408 4231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:47.397526 4231 leader_election.cc:302] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 924906c0df074b7285dec56100116f06; no voters:
I20260811 10:09:47.397792 4231 leader_election.cc:288] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 11 10:09:47 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:47.397858 4237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:09:47.398048 4237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 LEADER]: Becoming Leader. State: Replica: 924906c0df074b7285dec56100116f06, State: Running, Role: LEADER
I20260811 10:09:47.398149 4237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:47.398231 4231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:47.398517 4237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "924906c0df074b7285dec56100116f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } } }
I20260811 10:09:47.398603 4237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:47.398729 4237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 924906c0df074b7285dec56100116f06. Latest consensus state: current_term: 1 leader_uuid: "924906c0df074b7285dec56100116f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } } }
I20260811 10:09:47.398818 4237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:47.398988 4242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:09:47.399605 4242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:09:47.399888 4231 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 10:09:47.407521 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:47.391216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.22.177.62:40135)
0811 10:09:47.391481 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:47.391490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:47.391524 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:09:47.391968 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:47.391991 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:47.392067 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:47.392249 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:47.393217 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:47.394268 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:47.395064 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:47.395484 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:47.395610 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:47.398851 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:47.398855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:47.398856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:47.398883 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:47.404305 (+ 5422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:47.404929 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:47.404933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:47.404934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:47.405103 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:47.405348 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:47.405348 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:47.405348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:47.405902 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:47.406660 (+ 758us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:47.406739 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 10:09:47.407136 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":79,"threads_started":1}
I20260811 10:09:47.408380 4242 catalog_manager.cc:1347] Generated new cluster ID: d649d5eaa6ca4fa8928fd0a83513cb19
I20260811 10:09:47.408469 4242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:09:47.422745 4242 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:09:47.423642 4242 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:09:47.429350 4242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Generated new TSK 0
I20260811 10:09:47.429692 4242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:09:48.150137 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40135
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:48.243626 4279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:48.243884 4279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:48.243911 4279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:48.245201 4279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:48.245255 4279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:48.245285 4279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:09:48.247025 4279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.177.20:47933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:48.247298 4279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:48.247570 4279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:48.253964 4285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:48.254172 4286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:48.253947 4288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:48.255110 4279 server_base.cc:1034] running on GCE node
I20260811 10:09:48.255443 4279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:48.255748 4279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:48.256945 4279 hybrid_clock.cc:648] HybridClock initialized: now 1786442988256929 us; error 30 us; skew 500 ppm
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:48.260560 4279 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:09:48.261226 4279 webserver.cc:460] Webserver started at http://127.22.177.1:42253/ using document root <none> and password file <none>
I20260811 10:09:48.261518 4279 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:48.261600 4279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:48.261839 4279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:48.283650 4279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "a63087eb8dda436cb6de435f3d9d28e8"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "15ccd21ac92a5784a9f6ab261110e5e0"
server_key_iv: "d97cd08d43cc4efa6e3507699c728a3c"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.284268 4279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "a63087eb8dda436cb6de435f3d9d28e8"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "15ccd21ac92a5784a9f6ab261110e5e0"
server_key_iv: "d97cd08d43cc4efa6e3507699c728a3c"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.288568 4279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260811 10:09:48.303876 4295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.304107 4279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:48.304244 4279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "a63087eb8dda436cb6de435f3d9d28e8"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "15ccd21ac92a5784a9f6ab261110e5e0"
server_key_iv: "d97cd08d43cc4efa6e3507699c728a3c"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.304360 4279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:48.320504 4279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:48.325318 4279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:48.325572 4279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:48.325901 4279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:48.326342 4279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:48.326435 4279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.326538 4279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:48.326615 4279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.334235 4279 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:45779
I20260811 10:09:48.334789 4279 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:09:48.337558 4422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:45779 every 8 connection(s)
I20260811 10:09:48.342196 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 4279
I20260811 10:09:48.342332 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:09:48.342548 23236 external_mini_cluster.cc:1428] Setting key 3fe6f830e3007dae83dc810c3b3acfca
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:48.351588 4425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:48.338290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40135 (local address 127.22.177.1:55191)
0811 10:09:48.338668 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:48.338703 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:48.339181 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:48.339563 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:48.339567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:48.339875 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:48.340169 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:48.340176 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.341932 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.341934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:48.343514 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:48.343520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.344135 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.344136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:48.344309 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:48.345070 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:48.345082 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:48.347039 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:48.349531 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:48.349542 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:48.349545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:48.349955 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:48.350247 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:48.350248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:48.350248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:48.350420 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:48.350796 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:48.350803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:48.350991 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 10:09:48.351114 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 10:09:48.351310 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":83,"threads_started":1}
I20260811 10:09:48.352499 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:48.338292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:55191 (local address 127.22.177.62:40135)
0811 10:09:48.339340 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:48.339342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:48.339349 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:09:48.339371 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:48.339372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:48.339382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:48.339474 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:48.340420 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.341733 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.343674 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.343993 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.347176 (+ 3183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:48.347246 (+ 70us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:48.347247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:48.347247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:48.347253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:48.349392 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:48.350081 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:48.350082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:48.350082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:48.350164 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:48.350511 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:48.350512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:48.350512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:48.350691 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:48.350763 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:48.352277 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0811 10:09:48.352387 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018,"thread_start_us":101,"threads_started":1}
I20260811 10:09:48.352988 4423 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40135
I20260811 10:09:48.353152 4423 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:48.353367 4423 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:48.354035 4190 ts_manager.cc:194] Registered new tserver with Master: a63087eb8dda436cb6de435f3d9d28e8 (127.22.177.1:45779)
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
I20260811 10:09:48.354809 4190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:55191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:09:48.382547 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40135
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:48.508805 4429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:48.509117 4429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:48.509159 4429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:48.511058 4429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:48.511132 4429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:48.511174 4429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:09:48.513506 4429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.177.20:47933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:48.513826 4429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:48.514137 4429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:48.520983 4437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:48.521003 4438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:48.521256 4440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:48.522450 4429 server_base.cc:1034] running on GCE node
I20260811 10:09:48.522683 4429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:48.523023 4429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:48.524224 4429 hybrid_clock.cc:648] HybridClock initialized: now 1786442988524200 us; error 37 us; skew 500 ppm
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:48.527601 4429 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:09:48.528147 4429 webserver.cc:460] Webserver started at http://127.22.177.2:46643/ using document root <none> and password file <none>
I20260811 10:09:48.528522 4429 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:48.528594 4429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:48.528776 4429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:48.542980 4429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "f2077364b91948488d829727ad151274"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "cd6facc1a33ecec33e67a6cd0981a419"
server_key_iv: "96545cd6e07c324a455303c9bde20a47"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.543561 4429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "f2077364b91948488d829727ad151274"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "cd6facc1a33ecec33e67a6cd0981a419"
server_key_iv: "96545cd6e07c324a455303c9bde20a47"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.549504 4429 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260811 10:09:48.570755 4447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.571071 4429 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260811 10:09:48.571242 4429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "f2077364b91948488d829727ad151274"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "cd6facc1a33ecec33e67a6cd0981a419"
server_key_iv: "96545cd6e07c324a455303c9bde20a47"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.571521 4429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:48.588660 4429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:48.591219 4429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:48.591456 4429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:48.591773 4429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:48.592204 4429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:48.592315 4429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.592401 4429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:48.592465 4429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.599938 4429 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:39557
I20260811 10:09:48.600505 4429 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:09:48.601225 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 4429
I20260811 10:09:48.601337 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:09:48.601544 23236 external_mini_cluster.cc:1428] Setting key e74586eb8914e4e9144d8ce723ab8e33
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:09:48.617748 4574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:39557 every 8 connection(s)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:48.637244 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:48.618543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:43105 (local address 127.22.177.62:40135)
0811 10:09:48.618594 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:48.618597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:48.619097 (+ 500us) server_negotiation.cc:407] Connection header received
0811 10:09:48.619822 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:48.619824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:48.619843 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:48.619911 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:48.620716 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.621519 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.626053 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.626411 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.626465 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:48.629526 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:48.629528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:48.629528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:48.629538 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:48.632838 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:48.635706 (+ 2868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:48.635709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:48.635709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:48.635808 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:48.636264 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:48.636266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:48.636266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:48.636448 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:48.636506 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:48.637010 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 10:09:48.637138 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:09:48.638284 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:48.618536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40135 (local address 127.22.177.2:43105)
0811 10:09:48.618918 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:48.618973 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:48.619691 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:48.620028 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:48.620032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:48.620298 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:48.620592 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:48.620599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.621682 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.621683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:48.625904 (+ 4221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:48.625910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.626589 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.626590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:48.626777 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:48.627465 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:48.627476 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:48.629384 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:48.632987 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:48.633001 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:48.633004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:48.633424 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:48.635932 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:48.635933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:48.635934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:48.636146 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:48.636570 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:48.636581 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:48.636815 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 10:09:48.637774 (+ 959us) client_negotiation.cc:240] Negotiation successful
0811 10:09:48.638000 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":111,"threads_started":1}
I20260811 10:09:48.639132 4575 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40135
I20260811 10:09:48.639243 4575 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:48.639456 4575 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:48.639956 4190 ts_manager.cc:194] Registered new tserver with Master: f2077364b91948488d829727ad151274 (127.22.177.2:39557)
I20260811 10:09:48.640478 4190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:43105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:09:48.662225 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40135
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:48.795176 4584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:48.795511 4584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:48.795557 4584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:48.797437 4584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:48.797577 4584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:48.797695 4584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:09:48.800158 4584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.177.20:47933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:48.800470 4584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:48.800773 4584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:48.806849 4592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:48.807130 4589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:48.807453 4590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:48.807866 4584 server_base.cc:1034] running on GCE node
I20260811 10:09:48.808236 4584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:48.808579 4584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:48.811836 4584 hybrid_clock.cc:648] HybridClock initialized: now 1786442988811395 us; error 453 us; skew 500 ppm
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:48.816954 4584 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:09:48.817713 4584 webserver.cc:460] Webserver started at http://127.22.177.3:34715/ using document root <none> and password file <none>
I20260811 10:09:48.818085 4584 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:48.818245 4584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:48.818466 4584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:48.832437 4584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "31a53327ed5d46e8b0b3654823c7c7ae"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "a32c7ecb657afa4f8699d5d6bdbf90e1"
server_key_iv: "dc89ec2a6a2117f7103e1a4f2b591d3e"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.832984 4584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "31a53327ed5d46e8b0b3654823c7c7ae"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "a32c7ecb657afa4f8699d5d6bdbf90e1"
server_key_iv: "dc89ec2a6a2117f7103e1a4f2b591d3e"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.835156 4584 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 10:09:48.853811 4600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.854116 4584 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260811 10:09:48.854269 4584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "31a53327ed5d46e8b0b3654823c7c7ae"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "a32c7ecb657afa4f8699d5d6bdbf90e1"
server_key_iv: "dc89ec2a6a2117f7103e1a4f2b591d3e"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:48.854403 4584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:48.885840 4584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:48.888626 4584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:48.888877 4584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:48.889201 4584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:48.889689 4584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:48.889758 4584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.889796 4584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:48.889860 4584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:48.900010 4584 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:40887
I20260811 10:09:48.900543 4584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:09:48.902029 4729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:40887 every 8 connection(s)
I20260811 10:09:48.907665 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 4584
I20260811 10:09:48.907807 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:09:48.908036 23236 external_mini_cluster.cc:1428] Setting key 890654e14f50d065acb3fffc9795bacb
Aug 11 10:09:48 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:48.918421 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:48.902636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52755 (local address 127.22.177.62:40135)
0811 10:09:48.902692 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:48.902694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:48.905965 (+ 3271us) server_negotiation.cc:407] Connection header received
0811 10:09:48.906657 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:48.906659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:48.906674 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:48.906708 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:48.908708 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.909516 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.911137 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.911453 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.911504 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:48.914559 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:48.914561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:48.914561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:48.914570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:48.916569 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:48.917137 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:48.917138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:48.917139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:48.917217 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:48.917568 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:48.917569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:48.917569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:48.917747 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:48.917829 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:48.918215 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 10:09:48.918310 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:09:48.918651 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:48.902284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40135 (local address 127.22.177.3:52755)
0811 10:09:48.905768 (+ 3484us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:48.905831 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:48.906513 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:48.907071 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:48.907075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:48.907385 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:48.908601 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:48.908609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.909700 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.909701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:48.911021 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:48.911024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:48.911619 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:48.911620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:48.911785 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:48.912454 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:48.912465 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:48.914443 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:48.916676 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:48.916685 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:48.916688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:48.917024 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:48.917300 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:48.917301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:48.917301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:48.917470 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:48.917841 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:48.917848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:48.918029 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 10:09:48.918149 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 10:09:48.918325 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3418,"thread_start_us":115,"threads_started":1}
I20260811 10:09:48.919359 4730 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40135
I20260811 10:09:48.919467 4730 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:48.919660 4730 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:48.920060 4190 ts_manager.cc:194] Registered new tserver with Master: 31a53327ed5d46e8b0b3654823c7c7ae (127.22.177.3:40887)
I20260811 10:09:48.920564 4190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:52755
I20260811 10:09:48.930732 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:09:48.931770 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 4279
I20260811 10:09:48.949087 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 4429
I20260811 10:09:48.960573 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 4584
I20260811 10:09:48.966471 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 4154
I20260811 10:09:48.975221 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40135
--webserver_interface=127.22.177.62
--webserver_port=46351
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40135
--ranger_config_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:49.072419 4733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:49.072654 4733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:49.072678 4733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:49.074026 4733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:49.074077 4733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:49.074093 4733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:49.074108 4733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:49.074121 4733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:49.076586 4733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40135
--ranger_config_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=46351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:49.076967 4733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:49.077236 4733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:49.083076 4745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.083096 4740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.083096 4739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:49.083427 4733 server_base.cc:1034] running on GCE node
I20260811 10:09:49.083731 4733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:49.084049 4733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:49.085259 4733 hybrid_clock.cc:648] HybridClock initialized: now 1786442989085219 us; error 53 us; skew 500 ppm
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:49.088481 4733 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:09:49.089027 4733 webserver.cc:460] Webserver started at http://127.22.177.62:46351/ using document root <none> and password file <none>
I20260811 10:09:49.089296 4733 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:49.089383 4733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:49.174453 4733 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:09:49.175505 4754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.175782 4733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:49.175904 4733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "924906c0df074b7285dec56100116f06"
format_stamp: "Formatted at 2026-08-11 10:09:47 on dist-test-slave-g350"
server_key: "157f1117e8d6d1197ff67f398f71c8b1"
server_key_iv: "e652b5d4710a564b12f06b2b05a989e7"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:49.176383 4733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:49.198455 4733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:49.200685 4733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:49.200816 4733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:49.205487 4733 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40135
I20260811 10:09:49.205533 4812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40135 every 8 connection(s)
I20260811 10:09:49.205952 4733 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:09:49.207356 4813 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:09:49.208268 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Bootstrap starting.
I20260811 10:09:49.214200 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 4733
I20260811 10:09:49.214574 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:45779
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:40135
--webserver_port=42253
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:09:49.217929 4813 log.cc:824] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:49.220105 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:09:49.220494 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Bootstrap complete.
I20260811 10:09:49.223507 4813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:49.223791 4813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924906c0df074b7285dec56100116f06, State: Initialized, Role: FOLLOWER
I20260811 10:09:49.223990 4813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:49.224081 4813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:09:49.224149 4813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:09:49.224212 4813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:09:49.225443 4813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:49.225554 4813 leader_election.cc:302] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 924906c0df074b7285dec56100116f06; no voters:
I20260811 10:09:49.225878 4813 leader_election.cc:288] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:09:49.225925 4817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:09:49.226131 4817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [term 2 LEADER]: Becoming Leader. State: Replica: 924906c0df074b7285dec56100116f06, State: Running, Role: LEADER
I20260811 10:09:49.226233 4813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:49.226336 4817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } }
I20260811 10:09:49.226818 4818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "924906c0df074b7285dec56100116f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } } }
I20260811 10:09:49.226835 4819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 924906c0df074b7285dec56100116f06. Latest consensus state: current_term: 2 leader_uuid: "924906c0df074b7285dec56100116f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924906c0df074b7285dec56100116f06" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40135 } } }
I20260811 10:09:49.226946 4818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:49.226950 4819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:49.227435 4827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:09:49.228080 4827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:09:49.228324 4827 catalog_manager.cc:1259] Loaded cluster ID: d649d5eaa6ca4fa8928fd0a83513cb19
I20260811 10:09:49.228420 4827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:09:49.228499 4813 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 10:09:49.229873 4827 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:09:49.230130 4827 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Loaded TSK: 0
I20260811 10:09:49.230389 4827 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:09:49.319696 4815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:49.320113 4815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:49.320217 4815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:49.322150 4815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:49.322223 4815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:49.322265 4815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:09:49.324536 4815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:45779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=42253
--tserver_master_addrs=127.22.177.62:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:49.325069 4815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:49.325371 4815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:49.332528 4860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.333055 4858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.333256 4857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:49.333353 4815 server_base.cc:1034] running on GCE node
I20260811 10:09:49.333704 4815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:49.334033 4815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:49.335222 4815 hybrid_clock.cc:648] HybridClock initialized: now 1786442989335201 us; error 39 us; skew 500 ppm
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:49.339591 4815 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:09:49.340188 4815 webserver.cc:460] Webserver started at http://127.22.177.1:42253/ using document root <none> and password file <none>
I20260811 10:09:49.340502 4815 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:49.340615 4815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:49.368098 4815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:49.370453 4868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.370703 4815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 10:09:49.370814 4815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "a63087eb8dda436cb6de435f3d9d28e8"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "15ccd21ac92a5784a9f6ab261110e5e0"
server_key_iv: "d97cd08d43cc4efa6e3507699c728a3c"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:49.371349 4815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:49.392022 4815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:49.394984 4815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:49.395262 4815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:49.395560 4815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:49.395982 4815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:49.396039 4815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.396078 4815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:49.396162 4815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.403348 4815 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:45779
I20260811 10:09:49.404524 4815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:09:49.405287 5000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:45779 every 8 connection(s)
I20260811 10:09:49.408319 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 4815
I20260811 10:09:49.408660 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:39557
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:40135
--webserver_port=46643
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:49.420444 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.406288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40135 (local address 127.22.177.1:50881)
0811 10:09:49.406489 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:49.406540 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:49.407157 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:49.407720 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:49.407725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:49.408015 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:49.408299 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:49.408306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.410138 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.410139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:49.411346 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:49.411349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.412328 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.412329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:49.412460 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.413138 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:49.413149 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:49.415338 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:49.418364 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:49.418372 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:49.418374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:49.418707 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:49.419037 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:49.419038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:49.419038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:49.419183 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:49.419724 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:49.419730 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:49.419902 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 10:09:49.420013 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 10:09:49.420188 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
I20260811 10:09:49.421010 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.406608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:50881 (local address 127.22.177.62:40135)
0811 10:09:49.406883 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:49.406892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:49.406931 (+ 39us) server_negotiation.cc:407] Connection header received
0811 10:09:49.407427 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:49.407434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:49.407495 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:49.407649 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:49.408454 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.409997 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.411458 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.412088 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.412198 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.415482 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:49.415487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:49.415488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:49.415501 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:49.418255 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.418829 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.418833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.418834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.418952 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.419269 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.419271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.419271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.419605 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:49.419693 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:49.420657 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 10:09:49.420806 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260811 10:09:49.422055 5001 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40135
I20260811 10:09:49.422178 5001 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.422382 5001 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
W20260811 10:09:49.423377 5001 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:40135 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 10:09:49.423484 5001 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.423575 5001 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:49.423962 5001 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.424057 5001 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
W20260811 10:09:49.424389 5001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:09:49.527805 5006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:49.528051 5006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:49.528095 5006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:49.529707 5006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:49.529765 5006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:49.529795 5006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:09:49.531961 5006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:39557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=46643
--tserver_master_addrs=127.22.177.62:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:49.532305 5006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:49.532632 5006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:49.541340 5014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.541397 5011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.542764 5012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:49.543329 5006 server_base.cc:1034] running on GCE node
I20260811 10:09:49.543637 5006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:49.543965 5006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:49.545161 5006 hybrid_clock.cc:648] HybridClock initialized: now 1786442989545131 us; error 44 us; skew 500 ppm
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:49.548801 5006 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:09:49.549391 5006 webserver.cc:460] Webserver started at http://127.22.177.2:46643/ using document root <none> and password file <none>
I20260811 10:09:49.549710 5006 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:49.549822 5006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:49.582394 5006 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260811 10:09:49.584050 5023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.584290 5006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:49.584393 5006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "f2077364b91948488d829727ad151274"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "cd6facc1a33ecec33e67a6cd0981a419"
server_key_iv: "96545cd6e07c324a455303c9bde20a47"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:49.584908 5006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:49.601943 5006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:49.604676 5006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:49.604938 5006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:49.605247 5006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:49.605733 5006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:49.605846 5006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.605923 5006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:49.605981 5006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.612739 5006 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:39557
I20260811 10:09:49.613368 5006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:09:49.618337 5150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:39557 every 8 connection(s)
I20260811 10:09:49.619127 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5006
I20260811 10:09:49.619441 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.177.20:47933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:40887
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:40135
--webserver_port=34715
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:49.633217 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.615839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:46261 (local address 127.22.177.62:40135)
0811 10:09:49.615911 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:49.615913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:49.616286 (+ 373us) server_negotiation.cc:407] Connection header received
0811 10:09:49.617011 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:49.617013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:49.617030 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:49.617062 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:49.618002 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.618816 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.624418 (+ 5602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.624763 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.624850 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.628092 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:49.628095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:49.628095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:49.628105 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:49.631302 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.631944 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.631945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.631946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.632016 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.632313 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.632314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.632314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.632469 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:49.632525 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:49.632994 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 10:09:49.633109 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:09:49.633886 5153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.615529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40135 (local address 127.22.177.2:46261)
0811 10:09:49.616150 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:49.616196 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:49.616874 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:49.617259 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:49.617264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:49.617514 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:49.617839 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:49.617846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.620883 (+ 3037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.620884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:49.624267 (+ 3383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:49.624272 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.624948 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.624950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:49.625096 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.625803 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:49.625814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:49.627936 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:49.631437 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:49.631449 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:49.631453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:49.631837 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:49.632089 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:49.632089 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:49.632090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:49.632232 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:49.632568 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:49.632576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:49.632773 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 10:09:49.633386 (+ 613us) client_negotiation.cc:240] Negotiation successful
0811 10:09:49.633581 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":86,"threads_started":1}
I20260811 10:09:49.634670 5151 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40135
I20260811 10:09:49.634791 5151 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.635007 5151 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
W20260811 10:09:49.635749 5151 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:40135 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 10:09:49.635839 5151 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.635931 5151 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:49.636302 5151 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.636380 5151 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
W20260811 10:09:49.636741 5151 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:09:49.741657 5154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:49.741950 5154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:49.741989 5154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:49.743893 5154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:49.744057 5154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:49.744163 5154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:09:49.746789 5154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.177.20:47933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:40887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=34715
--tserver_master_addrs=127.22.177.62:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:49.747124 5154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:49.747505 5154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:49.754627 5163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.754858 5161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:49.754685 5160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:49.755792 5154 server_base.cc:1034] running on GCE node
I20260811 10:09:49.756067 5154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:49.756525 5154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:49.757691 5154 hybrid_clock.cc:648] HybridClock initialized: now 1786442989757667 us; error 36 us; skew 500 ppm
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:49.761869 5154 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:09:49.762467 5154 webserver.cc:460] Webserver started at http://127.22.177.3:34715/ using document root <none> and password file <none>
I20260811 10:09:49.762815 5154 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:49.762905 5154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:49.790189 5154 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.000s sys 0.002s
I20260811 10:09:49.791374 5173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.791646 5154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:49.791764 5154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "31a53327ed5d46e8b0b3654823c7c7ae"
format_stamp: "Formatted at 2026-08-11 10:09:48 on dist-test-slave-g350"
server_key: "a32c7ecb657afa4f8699d5d6bdbf90e1"
server_key_iv: "dc89ec2a6a2117f7103e1a4f2b591d3e"
server_key_version: "kuduclusterkey@0"
I20260811 10:09:49.792287 5154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:49.808290 5154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:49.811161 5154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:49.811352 5154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:49.811654 5154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:49.812077 5154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:49.812165 5154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.812220 5154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:49.812260 5154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:49.821219 5154 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:40887
I20260811 10:09:49.822456 5154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:09:49.822889 5302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:40887 every 8 connection(s)
Aug 11 10:09:49 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:49.830279 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5154
I20260811 10:09:49.834231 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.823478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:58787 (local address 127.22.177.62:40135)
0811 10:09:49.823547 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:49.823550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:49.823819 (+ 269us) server_negotiation.cc:407] Connection header received
0811 10:09:49.824400 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:49.824402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:49.824417 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:49.824480 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:49.825109 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.825765 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.827024 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.827399 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.827488 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.830466 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:49.830468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:49.830468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:49.830477 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:49.832237 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.832749 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.832751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.832751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.832845 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.833217 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.833218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.833218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.833355 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:49.833471 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:49.834033 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 10:09:49.834138 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:09:49.834414 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.823492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40135 (local address 127.22.177.3:58787)
0811 10:09:49.823689 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:49.823735 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:49.824279 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:49.824560 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:49.824565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:49.824780 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:49.824990 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:49.824995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.825879 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.825880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:49.826921 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:49.826924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.827532 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.827533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:49.827685 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.828489 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:49.828499 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:49.830291 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:49.832360 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:49.832370 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:49.832373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:49.832658 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:49.832943 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:49.832944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:49.832944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:49.833108 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:49.833666 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:49.833674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:49.833874 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 10:09:49.833994 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 10:09:49.834163 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260811 10:09:49.835186 5303 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40135
I20260811 10:09:49.835322 5303 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.835528 5303 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
W20260811 10:09:49.836344 5303 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:40135 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 10:09:49.836426 5303 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.836512 5303 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:49.836818 5303 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:49.836905 5303 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
W20260811 10:09:49.837188 5303 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:09:49.842633 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:49.830775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41510 (local address 127.22.177.62:40135)
0811 10:09:49.830952 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:49.830954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:49.833437 (+ 2483us) server_negotiation.cc:407] Connection header received
0811 10:09:49.833476 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:49.833477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:49.833493 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:49.833693 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:49.835582 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.836372 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.837473 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:49.837739 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:49.837797 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:49.839391 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:49.839392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:49.839392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:49.839399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:49.841106 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.841686 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.841687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.841688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.841757 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:49.842029 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:49.842030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:49.842031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:49.842181 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:49.842358 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:49.842427 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 10:09:49.842524 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260811 10:09:50.424587 5001 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:50.424774 5001 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:50.425246 4772 ts_manager.cc:194] Registered new tserver with Master: a63087eb8dda436cb6de435f3d9d28e8 (127.22.177.1:45779)
I20260811 10:09:50.426328 4772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:50881
Aug 11 10:09:50 dist-test-slave-g350 krb5kdc[3634](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786442990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:50.636925 5151 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:50.637112 5151 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:50.637535 4772 ts_manager.cc:194] Registered new tserver with Master: f2077364b91948488d829727ad151274 (127.22.177.2:39557)
I20260811 10:09:50.637985 4772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:46261
I20260811 10:09:50.837373 5303 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:50.837543 5303 heartbeater.cc:511] Master 127.22.177.62:40135 requested a full tablet report, sending...
I20260811 10:09:50.838022 4772 ts_manager.cc:194] Registered new tserver with Master: 31a53327ed5d46e8b0b3654823c7c7ae (127.22.177.3:40887)
I20260811 10:09:50.838562 4772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:58787
I20260811 10:09:50.840075 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:09:50.852380 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.841157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.22.177.62:40135)
0811 10:09:50.841327 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:50.841330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:50.841552 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:09:50.841584 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:50.841609 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:50.841625 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:50.841714 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:50.842415 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.843193 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.844346 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.844651 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.844755 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.846848 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:50.846850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:50.846850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:50.846858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:50.850598 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:50.851045 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:50.851046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:50.851047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:50.851113 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:50.851324 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:50.851325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:50.851325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:50.851460 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:50.851542 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:50.852059 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 10:09:50.852226 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":73,"threads_started":1}
I20260811 10:09:50.860921 4772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:09:50.864043 4772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:09:50.875250 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.868378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39557 (local address 127.0.0.1:57612)
0811 10:09:50.869388 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:50.869399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:50.869443 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:50.869857 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:50.869860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:50.869863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:50.869941 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:50.869943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.871907 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.871908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:50.873723 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:50.873727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.874901 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.874902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:50.874988 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.875003 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:09:50.875083 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:09:50.875153 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":97,"threads_started":1}
I20260811 10:09:50.876286 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.868660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.22.177.2:39557)
0811 10:09:50.869558 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:50.869564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:50.869574 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:09:50.869636 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:50.869638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:50.869666 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:50.869745 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:50.870040 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.871795 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.873831 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.874698 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.874783 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.874859 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:50.875786 (+ 927us) server_negotiation.cc:299] Negotiation successful
0811 10:09:50.875840 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"thread_start_us":53,"threads_started":1}
I20260811 10:09:50.877251 5054 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 (DEFAULT_TABLE table=test-table [id=e217115ef7454a868b671fad1b6c8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:50.877792 5054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 10:09:50 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:50.883287 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.868375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:40887 (local address 127.0.0.1:46172)
0811 10:09:50.869137 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:50.869186 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:50.869260 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:50.876883 (+ 7623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:50.876884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:50.876888 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:50.876939 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:50.876941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.878641 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.878642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:50.880058 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:50.880061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.883000 (+ 2939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.883001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:50.883090 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.883097 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:09:50.883134 (+ 37us) client_negotiation.cc:240] Negotiation successful
0811 10:09:50.883189 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":89,"threads_started":1}
I20260811 10:09:50.883860 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.868792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.22.177.3:40887)
0811 10:09:50.876663 (+ 7871us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:50.876670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:50.876684 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:09:50.876727 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:50.876730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:50.876750 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:50.877029 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:50.877129 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.878270 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.880174 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.882227 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.883432 (+ 1205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.883517 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:50.883553 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 10:09:50.883771 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7809,"thread_start_us":64,"threads_started":1}
I20260811 10:09:50.884594 5222 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 (DEFAULT_TABLE table=test-table [id=e217115ef7454a868b671fad1b6c8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:50.884979 5222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:50.886811 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.868611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:45779 (local address 127.0.0.1:35136)
0811 10:09:50.871139 (+ 2528us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:50.871151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:50.871206 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:50.880835 (+ 9629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:50.880836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:50.880840 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:50.880960 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:50.880962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.884008 (+ 3046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.884009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:50.885313 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:50.885317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.886508 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.886510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:50.886589 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.886595 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:09:50.886629 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 10:09:50.886721 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2507,"thread_start_us":94,"threads_started":1}
I20260811 10:09:50.887460 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:50.869002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.22.177.1:45779)
0811 10:09:50.878437 (+ 9435us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:50.878442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:50.878454 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:09:50.878490 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:50.878492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:50.878510 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:50.881059 (+ 2549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:50.881166 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.882865 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.885443 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:50.886376 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:50.887207 (+ 831us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:50.887288 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:50.887321 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 10:09:50.887371 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9391,"thread_start_us":43,"threads_started":1}
I20260811 10:09:50.888346 4919 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 (DEFAULT_TABLE table=test-table [id=e217115ef7454a868b671fad1b6c8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:50.888744 4919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:50.893038 5342 tablet_bootstrap.cc:492] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: Bootstrap starting.
I20260811 10:09:50.894014 5342 tablet_bootstrap.cc:654] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:50.894291 5340 tablet_bootstrap.cc:492] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Bootstrap starting.
I20260811 10:09:50.895150 5340 tablet_bootstrap.cc:654] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:50.895610 5342 log.cc:824] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:50.896306 5340 log.cc:824] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:50.896764 5341 tablet_bootstrap.cc:492] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Bootstrap starting.
I20260811 10:09:50.897831 5341 tablet_bootstrap.cc:654] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:50.898267 5341 log.cc:824] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:50.898984 5342 tablet_bootstrap.cc:492] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: No bootstrap required, opened a new log
I20260811 10:09:50.899091 5342 ts_tablet_manager.cc:1397] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 10:09:50.901240 5342 raft_consensus.cc:348] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:50.901445 5342 raft_consensus.cc:374] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:50.901968 5340 tablet_bootstrap.cc:492] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: No bootstrap required, opened a new log
I20260811 10:09:50.902087 5340 ts_tablet_manager.cc:1397] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260811 10:09:50.904225 5340 raft_consensus.cc:348] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:50.904417 5340 raft_consensus.cc:374] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:50.904536 5340 raft_consensus.cc:729] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2077364b91948488d829727ad151274, State: Initialized, Role: FOLLOWER
I20260811 10:09:50.904518 5342 raft_consensus.cc:729] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63087eb8dda436cb6de435f3d9d28e8, State: Initialized, Role: FOLLOWER
I20260811 10:09:50.904893 5340 consensus_queue.cc:260] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:50.905210 5341 tablet_bootstrap.cc:492] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: No bootstrap required, opened a new log
I20260811 10:09:50.905308 5341 ts_tablet_manager.cc:1397] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260811 10:09:50.905452 5340 ts_tablet_manager.cc:1428] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:09:50.905768 5342 consensus_queue.cc:260] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:50.905921 5151 heartbeater.cc:503] Master 127.22.177.62:40135 was elected leader, sending a full tablet report...
I20260811 10:09:50.906250 5342 ts_tablet_manager.cc:1428] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 10:09:50.906421 5001 heartbeater.cc:503] Master 127.22.177.62:40135 was elected leader, sending a full tablet report...
W20260811 10:09:50.906831 5002 tablet.cc:2369] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:09:50.908022 5341 raft_consensus.cc:348] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:50.908262 5341 raft_consensus.cc:374] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:50.908327 5341 raft_consensus.cc:729] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a53327ed5d46e8b0b3654823c7c7ae, State: Initialized, Role: FOLLOWER
I20260811 10:09:50.908481 5341 consensus_queue.cc:260] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:50.908807 5341 ts_tablet_manager.cc:1428] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 10:09:50.908862 5303 heartbeater.cc:503] Master 127.22.177.62:40135 was elected leader, sending a full tablet report...
W20260811 10:09:51.073797 5304 tablet.cc:2369] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:09:51.117090 5152 tablet.cc:2369] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:09:51.191785 5348 raft_consensus.cc:482] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:09:51.191947 5348 raft_consensus.cc:504] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:51.192394 5348 leader_election.cc:288] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2077364b91948488d829727ad151274 (127.22.177.2:39557), a63087eb8dda436cb6de435f3d9d28e8 (127.22.177.1:45779)
I20260811 10:09:51.196810 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:51.192599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39557 (local address 127.22.177.3:41077)
0811 10:09:51.193147 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:51.193160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:51.193251 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:51.193438 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:51.193439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:51.193440 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:51.193515 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:51.193517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.194549 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.194550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:51.195712 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:51.195716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.196521 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.196522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:51.196586 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:51.196590 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:09:51.196633 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 10:09:51.196683 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":123,"threads_started":1}
I20260811 10:09:51.196813 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:51.192673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:41077 (local address 127.22.177.2:39557)
0811 10:09:51.192728 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:51.192730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:51.193220 (+ 490us) server_negotiation.cc:407] Connection header received
0811 10:09:51.193328 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:51.193329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:51.193341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:51.193370 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:51.193583 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.194316 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.195816 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.196353 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.196400 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:51.196409 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:51.196690 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 10:09:51.196745 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:09:51.197569 5086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d3ea9109d4c5fb3a1539a781dfbd2" candidate_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2077364b91948488d829727ad151274" is_pre_election: true
I20260811 10:09:51.197818 5086 raft_consensus.cc:2454] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a53327ed5d46e8b0b3654823c7c7ae in term 0.
I20260811 10:09:51.198087 5176 leader_election.cc:302] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a53327ed5d46e8b0b3654823c7c7ae, f2077364b91948488d829727ad151274; no voters:
I20260811 10:09:51.198236 5348 raft_consensus.cc:2781] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:09:51.198284 5348 raft_consensus.cc:482] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:09:51.198308 5348 raft_consensus.cc:3037] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:51.199288 5348 raft_consensus.cc:504] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:51.199434 5348 leader_election.cc:288] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [CANDIDATE]: Term 1 election: Requested vote from peers f2077364b91948488d829727ad151274 (127.22.177.2:39557), a63087eb8dda436cb6de435f3d9d28e8 (127.22.177.1:45779)
I20260811 10:09:51.199632 5086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d3ea9109d4c5fb3a1539a781dfbd2" candidate_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2077364b91948488d829727ad151274"
I20260811 10:09:51.199712 5086 raft_consensus.cc:3037] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:51.200407 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:51.193004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:45299 (local address 127.22.177.1:45779)
0811 10:09:51.193058 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:51.193061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:51.197266 (+ 4205us) server_negotiation.cc:407] Connection header received
0811 10:09:51.197367 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:51.197368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:51.197381 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:51.197410 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:51.197627 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.198317 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.199517 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.200036 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.200084 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:51.200092 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:51.200294 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 10:09:51.200340 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:09:51.200590 5086 raft_consensus.cc:2454] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a53327ed5d46e8b0b3654823c7c7ae in term 1.
I20260811 10:09:51.200613 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:51.192823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:45779 (local address 127.22.177.3:45299)
0811 10:09:51.197202 (+ 4379us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:51.197213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:51.197297 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:51.197478 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:51.197479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:51.197480 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:51.197511 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:51.197512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.198419 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.198421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:51.199412 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:51.199415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.200149 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.200150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:51.200202 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:51.200204 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:09:51.200513 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 10:09:51.200554 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4366,"thread_start_us":67,"threads_started":1}
I20260811 10:09:51.200789 5176 leader_election.cc:302] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a53327ed5d46e8b0b3654823c7c7ae, f2077364b91948488d829727ad151274; no voters:
I20260811 10:09:51.200901 5348 raft_consensus.cc:2781] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:09:51.201066 5348 raft_consensus.cc:686] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 1 LEADER]: Becoming Leader. State: Replica: 31a53327ed5d46e8b0b3654823c7c7ae, State: Running, Role: LEADER
I20260811 10:09:51.201156 5348 consensus_queue.cc:237] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } }
I20260811 10:09:51.201143 4945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d3ea9109d4c5fb3a1539a781dfbd2" candidate_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a63087eb8dda436cb6de435f3d9d28e8" is_pre_election: true
I20260811 10:09:51.201299 4945 raft_consensus.cc:2454] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a53327ed5d46e8b0b3654823c7c7ae in term 0.
I20260811 10:09:51.201432 4944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d3ea9109d4c5fb3a1539a781dfbd2" candidate_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a63087eb8dda436cb6de435f3d9d28e8"
I20260811 10:09:51.201502 4944 raft_consensus.cc:3037] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:51.202026 4772 catalog_manager.cc:5495] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a53327ed5d46e8b0b3654823c7c7ae (127.22.177.3). New cstate: current_term: 1 leader_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a53327ed5d46e8b0b3654823c7c7ae" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 } health_report { overall_health: UNKNOWN } } }
I20260811 10:09:51.202390 4944 raft_consensus.cc:2454] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a53327ed5d46e8b0b3654823c7c7ae in term 1.
Aug 11 10:09:51 dist-test-slave-g350 krb5kdc[3634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for HTTP/127.22.177.20@KRBTEST.COM
I20260811 10:09:51.229158 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:51.225538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.22.177.3:40887)
0811 10:09:51.225673 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:51.225675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:51.225684 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:09:51.225776 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:51.225777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:51.225800 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:51.225865 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:09:51.226056 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.226766 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.228009 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:51.228295 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:51.228363 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:51.228579 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:09:51.228722 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:09:51.228805 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:51.229053 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 10:09:51.229077 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 10:09:51.231389 4944 raft_consensus.cc:1261] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P a63087eb8dda436cb6de435f3d9d28e8 [term 1 FOLLOWER]: Refusing update from remote peer 31a53327ed5d46e8b0b3654823c7c7ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:09:51.231765 5352 consensus_queue.cc:1035] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a63087eb8dda436cb6de435f3d9d28e8" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:09:51.234333 5086 raft_consensus.cc:1261] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 1 FOLLOWER]: Refusing update from remote peer 31a53327ed5d46e8b0b3654823c7c7ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:09:51.234624 5352 consensus_queue.cc:1035] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2077364b91948488d829727ad151274" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:09:51.238502 5355 mvcc.cc:205] Tried to move back new op lower bound from 7317270492080427008 to 7317270491960418304. Current Snapshot: MvccSnapshot[applied={T|T < 7317270492080427008}]
I20260811 10:09:51.239140 5358 mvcc.cc:205] Tried to move back new op lower bound from 7317270492080427008 to 7317270491960418304. Current Snapshot: MvccSnapshot[applied={T|T < 7317270492080427008}]
I20260811 10:09:51.241178 5356 mvcc.cc:205] Tried to move back new op lower bound from 7317270492080427008 to 7317270491960418304. Current Snapshot: MvccSnapshot[applied={T|T < 7317270492080427008 or (T in {7317270492080427008})}]
I20260811 10:09:51.243899 4772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:09:51.244009 4772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:09:51.244925 4772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 924906c0df074b7285dec56100116f06: Sending DeleteTablet for 3 replicas of tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2
I20260811 10:09:51.245288 5222 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:09:51 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:46172
I20260811 10:09:51.245837 5054 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:09:51 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:57612
I20260811 10:09:51.246126 4919 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:09:51 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:35136
I20260811 10:09:51.246944 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 4815
I20260811 10:09:51.255192 5366 tablet_replica.cc:330] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: stopping tablet replica
I20260811 10:09:51.255389 5366 raft_consensus.cc:2229] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:09:51.255558 5366 raft_consensus.cc:2258] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:09:51.255771 5364 tablet_replica.cc:330] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: stopping tablet replica
I20260811 10:09:51.255915 5364 raft_consensus.cc:2229] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:09:51.256067 5364 raft_consensus.cc:2258] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:09:51.256744 5364 ts_tablet_manager.cc:1905] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:09:51.259059 5366 ts_tablet_manager.cc:1905] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:09:51.260850 5366 ts_tablet_manager.cc:1918] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:09:51.260934 5366 log.cc:1196] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/2e3d3ea9109d4c5fb3a1539a781dfbd2
I20260811 10:09:51.261227 5366 ts_tablet_manager.cc:1939] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P f2077364b91948488d829727ad151274: Deleting consensus metadata
I20260811 10:09:51.266589 5364 ts_tablet_manager.cc:1918] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:09:51.266716 5364 log.cc:1196] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/2e3d3ea9109d4c5fb3a1539a781dfbd2
I20260811 10:09:51.267040 5364 ts_tablet_manager.cc:1939] T 2e3d3ea9109d4c5fb3a1539a781dfbd2 P 31a53327ed5d46e8b0b3654823c7c7ae: Deleting consensus metadata
I20260811 10:09:51.267572 4758 catalog_manager.cc:4841] TS 31a53327ed5d46e8b0b3654823c7c7ae (127.22.177.3:40887): tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 (table test-table [id=e217115ef7454a868b671fad1b6c8a7d]) successfully deleted
I20260811 10:09:51.268092 4757 catalog_manager.cc:4841] TS f2077364b91948488d829727ad151274 (127.22.177.2:39557): tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2 (table test-table [id=e217115ef7454a868b671fad1b6c8a7d]) successfully deleted
W20260811 10:09:51.273910 4757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:45779 (error 108)
I20260811 10:09:51.278049 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5006
I20260811 10:09:51.285545 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:51.285128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:45779 (local address 127.0.0.1:35138)
0811 10:09:51.285406 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:51.285488 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:45779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 10:09:51.285745 4757 catalog_manager.cc:4584] TS a63087eb8dda436cb6de435f3d9d28e8 (127.22.177.1:45779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e3d3ea9109d4c5fb3a1539a781dfbd2: Network error: Client connection negotiation failed: client connection to 127.22.177.1:45779: connect: Connection refused (error 111)
I20260811 10:09:51.292695 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5154
I20260811 10:09:51.303942 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 4733
I20260811 10:09:51.315831 23236 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 10:09:51.660969 23236 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 10:09:51.661083 23236 mini_ranger.cc:66] Stopping Ranger...
I20260811 10:09:52.016626 23236 mini_ranger.cc:68] Stopped Ranger
2026-08-11 10:09:52.016 UTC [3667] LOG: received smart shutdown request
2026-08-11 10:09:52.021 UTC [3667] LOG: background worker "logical replication launcher" (PID 3676) exited with exit code 1
2026-08-11 10:09:52.022 UTC [3671] LOG: shutting down
2026-08-11 10:09:52.219 UTC [3667] LOG: database system is shut down
2026-08-11T10:09:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55576 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:09:52 dist-test-slave-g350 krb5kdc[5380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:09:52 dist-test-slave-g350 krb5kdc[5380](info): set up 2 sockets
Aug 11 10:09:52 dist-test-slave-g350 krb5kdc[5380](info): commencing operation
krb5kdc: starting...
W20260811 10:09:53.469501 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:09:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:09:53.614295 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:34507
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:34507,127.22.177.61:38029,127.22.177.60:33419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:53.700397 5395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:53.700671 5395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:53.700709 5395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:53.701956 5395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:53.702045 5395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:53.702075 5395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:53.702103 5395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:53.702131 5395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:53.703791 5395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:34507,127.22.177.61:38029,127.22.177.60:33419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:34507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:53.704121 5395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:53.704381 5395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:53.710248 5400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:53.710247 5404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:53.710098 5401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:53.710510 5395 server_base.cc:1034] running on GCE node
I20260811 10:09:53.710816 5395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:53.711107 5395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:53.712301 5395 hybrid_clock.cc:648] HybridClock initialized: now 1786442993712314 us; error 70 us; skew 500 ppm
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:53.715140 5395 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:09:53.715633 5395 webserver.cc:460] Webserver started at http://127.22.177.62:33945/ using document root <none> and password file <none>
I20260811 10:09:53.715885 5395 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:53.715932 5395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:53.716028 5395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:53.717173 5395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "372c2505522947638e7355bf1701be72"
format_stamp: "Formatted at 2026-08-11 10:09:53 on dist-test-slave-g350"
server_key: "29155300160f4f558fcf776b9f318cc3"
server_key_iv: "23cb278a48e9843568e2525a52074887"
server_key_version: "encryptionkey@0"
I20260811 10:09:53.717564 5395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "372c2505522947638e7355bf1701be72"
format_stamp: "Formatted at 2026-08-11 10:09:53 on dist-test-slave-g350"
server_key: "29155300160f4f558fcf776b9f318cc3"
server_key_iv: "23cb278a48e9843568e2525a52074887"
server_key_version: "encryptionkey@0"
I20260811 10:09:53.719105 5395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:53.720286 5411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:53.720520 5395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:53.720619 5395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "372c2505522947638e7355bf1701be72"
format_stamp: "Formatted at 2026-08-11 10:09:53 on dist-test-slave-g350"
server_key: "29155300160f4f558fcf776b9f318cc3"
server_key_iv: "23cb278a48e9843568e2525a52074887"
server_key_version: "encryptionkey@0"
I20260811 10:09:53.720679 5395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:53.730151 5395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:53.732905 5395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:53.733042 5395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:53.737321 5395 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:34507
I20260811 10:09:53.737401 5468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:34507 every 8 connection(s)
I20260811 10:09:53.737982 5395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:09:53.738838 5469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:53.740797 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5395
I20260811 10:09:53.740927 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:09:53.740705 5469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:09:53.741144 23236 external_mini_cluster.cc:1428] Setting key 033f792a3c25657fa5e55d41b51ba6e9
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.61@KRBTEST.COM; defaulting to no policy
I20260811 10:09:53.752084 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.741569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:34507 (local address 127.0.0.1:51772)
0811 10:09:53.741811 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.741859 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:53.742584 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:53.742970 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:53.742973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:53.743347 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:53.743509 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.743513 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.744609 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.744610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.745315 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.745317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.745923 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.745924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.745990 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.746591 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:53.746599 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:53.748067 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:53.750392 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.750400 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.750403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.750625 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.750905 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.750906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.750906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.751004 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.751407 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:53.751412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:53.751480 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 10:09:53.751597 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:09:53.751858 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":98,"thread_start_us":132,"threads_started":1}
I20260811 10:09:53.752082 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.741631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.22.177.62:34507)
0811 10:09:53.741832 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:53.741841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:53.741959 (+ 118us) server_negotiation.cc:407] Connection header received
0811 10:09:53.742819 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:53.742821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:53.742839 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:53.742934 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:53.743631 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.744511 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.745404 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.745824 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.745950 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.748159 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:53.748167 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:53.748168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:53.748175 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:53.750274 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.750731 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.750734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.750735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.750829 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.751075 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.751077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.751077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.751329 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:53.751432 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:53.751626 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 10:09:53.751786 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":66,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.61@KRBTEST.COM" created.
I20260811 10:09:53.753084 5469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:09:53.753473 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.753345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49138)
0811 10:09:53.753395 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.753443 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 10:09:53.753659 5414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
I20260811 10:09:53.754088 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.754017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49146)
0811 10:09:53.754045 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.754070 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260811 10:09:53.754205 5469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:38029: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab.
Entry for principal kudu/127.22.177.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab.
I20260811 10:09:53.773844 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:38029
--webserver_interface=127.22.177.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab
--principal=kudu/127.22.177.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:34507,127.22.177.61:38029,127.22.177.60:33419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:09:53.803486 5469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } attempt: 1
I20260811 10:09:53.804092 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.803941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49150)
0811 10:09:53.804014 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.804058 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 10:09:53.804570 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.804429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49152)
0811 10:09:53.804501 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.804535 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 10:09:53.804704 5469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:38029: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
W20260811 10:09:53.866724 5477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:53.866961 5477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:53.867000 5477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:53.868496 5477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:53.868570 5477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:53.868597 5477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:53.868628 5477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:53.868654 5477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:53.870412 5477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:34507,127.22.177.61:38029,127.22.177.60:33419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab
--principal=kudu/127.22.177.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.177.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:53.870723 5477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:53.870972 5477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:09:53.872838 5469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } attempt: 2
I20260811 10:09:53.873284 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.873144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49154)
0811 10:09:53.873213 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.873249 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:09:53.873624 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.873508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49166)
0811 10:09:53.873551 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.873577 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 10:09:53.873704 5469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:38029: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38029: connect: Connection refused (error 111)
W20260811 10:09:53.876405 5488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:53.876420 5485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:53.876367 5486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:53.876910 5477 server_base.cc:1034] running on GCE node
I20260811 10:09:53.877189 5477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:53.877508 5477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:53.878688 5477 hybrid_clock.cc:648] HybridClock initialized: now 1786442993878676 us; error 48 us; skew 500 ppm
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:53.881337 5477 init.cc:377] Logged in from keytab as kudu/127.22.177.61@KRBTEST.COM (short username kudu)
I20260811 10:09:53.881902 5477 webserver.cc:460] Webserver started at http://127.22.177.61:39645/ using document root <none> and password file <none>
I20260811 10:09:53.882155 5477 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:53.882225 5477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:53.882373 5477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:53.883533 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data/instance:
uuid: "1f76b46f55e841c99afdfb1fefcd0d73"
format_stamp: "Formatted at 2026-08-11 10:09:53 on dist-test-slave-g350"
server_key: "40b6537fa9c9554dbbef313c541ede99"
server_key_iv: "560228fbaee2dd6704e64e6f1ab84d57"
server_key_version: "encryptionkey@0"
I20260811 10:09:53.883957 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal/instance:
uuid: "1f76b46f55e841c99afdfb1fefcd0d73"
format_stamp: "Formatted at 2026-08-11 10:09:53 on dist-test-slave-g350"
server_key: "40b6537fa9c9554dbbef313c541ede99"
server_key_iv: "560228fbaee2dd6704e64e6f1ab84d57"
server_key_version: "encryptionkey@0"
I20260811 10:09:53.885505 5477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:09:53.886384 5495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:53.886624 5477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:53.886754 5477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal
uuid: "1f76b46f55e841c99afdfb1fefcd0d73"
format_stamp: "Formatted at 2026-08-11 10:09:53 on dist-test-slave-g350"
server_key: "40b6537fa9c9554dbbef313c541ede99"
server_key_iv: "560228fbaee2dd6704e64e6f1ab84d57"
server_key_version: "encryptionkey@0"
I20260811 10:09:53.886859 5477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:53.937979 5477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:53.946446 5477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:53.946679 5477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:53.955881 5477 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.61:38029
I20260811 10:09:53.955994 5558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.61:38029 every 8 connection(s)
I20260811 10:09:53.956351 5477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
I20260811 10:09:53.957598 5560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:53.959860 5560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:09:53.964847 5469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } attempt: 3
I20260811 10:09:53.966178 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5477
I20260811 10:09:53.966354 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-1/wal/instance
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:53.966557 23236 external_mini_cluster.cc:1428] Setting key 6a9c795583e37f6791c51b167e34f4b3
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:09:53.970721 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.960788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51778 (local address 127.22.177.62:34507)
0811 10:09:53.960862 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:53.960865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:53.961088 (+ 223us) server_negotiation.cc:407] Connection header received
0811 10:09:53.961702 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:53.961704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:53.961727 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:53.961779 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:53.962443 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.962958 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.963955 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.964186 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.964262 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.967148 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:53.967149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:53.967150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:53.967158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:53.969025 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.969502 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.969503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.969503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.969579 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.969949 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.969950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.969951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.970098 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:53.970143 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:53.970535 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 10:09:53.970619 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:09:53.970940 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.960765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:34507 (local address 127.0.0.1:51778)
0811 10:09:53.960961 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.961000 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:53.961559 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:53.961848 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:53.961852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:53.962123 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:53.962346 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.962351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.963059 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.963060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.963862 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.963864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.964270 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.964271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.964408 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.965072 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:53.965082 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:53.967057 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:53.969112 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.969121 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.969123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.969414 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.969708 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.969709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.969709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.969855 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.970195 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:53.970217 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:53.970385 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 10:09:53.970515 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 10:09:53.970677 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260811 10:09:53.971655 5560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:09:53.975430 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.965143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49172)
0811 10:09:53.965197 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.965208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:53.965254 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:53.965619 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:53.965621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:53.965808 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:53.965859 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.965861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.966754 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.966754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.967382 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.967385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.967987 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.967988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.968070 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.968604 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:53.968610 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:53.970329 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:53.974155 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.974156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.974156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.974404 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.974635 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.974636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.974636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.974714 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.975083 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:53.975085 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:53.975142 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 10:09:53.975221 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 10:09:53.975360 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 10:09:53.975485 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.965236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.22.177.61:38029)
0811 10:09:53.965452 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:53.965457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:53.965474 (+ 17us) server_negotiation.cc:407] Connection header received
0811 10:09:53.965493 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:53.965494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:53.965510 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:53.965597 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:53.965945 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.966622 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.967499 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.967890 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.967986 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.970429 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:53.970437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:53.970438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:53.970446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:53.974060 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.974497 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.974500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.974501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.974568 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.974803 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.974804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.974804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.975001 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:53.975098 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:53.975277 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 10:09:53.975361 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":127,"threads_started":1}
I20260811 10:09:53.976379 5469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:09:53.976799 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.976682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56302)
0811 10:09:53.976732 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.976761 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
Aug 11 10:09:53 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.60@KRBTEST.COM; defaulting to no policy
I20260811 10:09:53.977139 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.977066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56310)
0811 10:09:53.977096 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.977119 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260811 10:09:53.977240 5469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:33419: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.60@KRBTEST.COM" created.
I20260811 10:09:53.980656 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.971984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49174)
0811 10:09:53.972028 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.972038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:53.972075 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:53.972307 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:53.972307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:53.972469 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:53.972525 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.972526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.973514 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.973515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.974298 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:53.974300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.974776 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.974777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:53.974846 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.975584 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:53.975590 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:53.977245 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:53.979495 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.979496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.979497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.979691 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.979925 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:53.979926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:53.979926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:53.979986 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:53.980273 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:53.980274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:53.980326 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 10:09:53.980455 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 10:09:53.980584 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 10:09:53.980832 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.972033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.22.177.61:38029)
0811 10:09:53.972199 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:53.972200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:53.972208 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:09:53.972222 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:53.972223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:53.972230 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:53.972306 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:53.972606 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.973421 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.974387 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:53.974691 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:53.974772 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:53.977354 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:53.977355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:53.977356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:53.977363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:53.979375 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.979788 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.979789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.979789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.979850 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:53.980047 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:53.980048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:53.980048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:53.980190 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:53.980265 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:53.980491 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 10:09:53.980662 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
I20260811 10:09:53.981171 5560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:09:53.981500 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.981397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56320)
0811 10:09:53.981432 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.981481 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260811 10:09:53.981719 5499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
I20260811 10:09:53.982184 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:53.982095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56322)
0811 10:09:53.982137 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:53.982164 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 10:09:53.982316 5560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:33419: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab.
Entry for principal kudu/127.22.177.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab.
I20260811 10:09:54.003471 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:33419
--webserver_interface=127.22.177.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab
--principal=kudu/127.22.177.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:34507,127.22.177.61:38029,127.22.177.60:33419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:09:54.008526 5469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } attempt: 1
I20260811 10:09:54.009083 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.008926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56330)
0811 10:09:54.008997 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.009048 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 10:09:54.009485 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.009371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56344)
0811 10:09:54.009427 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.009461 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 10:09:54.009577 5469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:33419: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
I20260811 10:09:54.031594 5560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } attempt: 1
I20260811 10:09:54.032159 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.031976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56352)
0811 10:09:54.032058 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.032111 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:09:54.032557 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.032461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56358)
0811 10:09:54.032503 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.032537 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 10:09:54.032704 5560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:33419: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
I20260811 10:09:54.084930 5469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } attempt: 2
I20260811 10:09:54.085531 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.085361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56366)
0811 10:09:54.085453 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.085498 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 10:09:54.085950 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.085851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56370)
0811 10:09:54.085902 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.085931 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 10:09:54.086064 5469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:33419: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
W20260811 10:09:54.094187 5569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:54.094422 5569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:54.094445 5569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:54.095605 5569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:54.095645 5569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:54.095660 5569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:54.095674 5569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:54.095690 5569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:54.097302 5569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:34507,127.22.177.61:38029,127.22.177.60:33419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab
--principal=kudu/127.22.177.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:33419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.177.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:54.097527 5569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:54.097805 5569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:09:54.100905 5560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } attempt: 2
I20260811 10:09:54.101385 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.101220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56386)
0811 10:09:54.101320 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.101356 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 10:09:54.101917 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.101762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56402)
0811 10:09:54.101839 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.101867 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 10:09:54.102137 5560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:33419: Network error: Client connection negotiation failed: client connection to 127.22.177.60:33419: connect: Connection refused (error 111)
W20260811 10:09:54.103430 5575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:54.103431 5578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:54.103504 5576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:54.103716 5569 server_base.cc:1034] running on GCE node
I20260811 10:09:54.104007 5569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:54.104311 5569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:54.105463 5569 hybrid_clock.cc:648] HybridClock initialized: now 1786442994105422 us; error 53 us; skew 500 ppm
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442994, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:54.108901 5569 init.cc:377] Logged in from keytab as kudu/127.22.177.60@KRBTEST.COM (short username kudu)
I20260811 10:09:54.109375 5569 webserver.cc:460] Webserver started at http://127.22.177.60:38919/ using document root <none> and password file <none>
I20260811 10:09:54.109711 5569 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:54.109783 5569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:54.109931 5569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:54.111079 5569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data/instance:
uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a"
format_stamp: "Formatted at 2026-08-11 10:09:54 on dist-test-slave-g350"
server_key: "9e849262b2ad4eb249df2853b861f5c1"
server_key_iv: "c9b5e06e070cf102805dd7db4eae8c14"
server_key_version: "encryptionkey@0"
I20260811 10:09:54.111456 5569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal/instance:
uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a"
format_stamp: "Formatted at 2026-08-11 10:09:54 on dist-test-slave-g350"
server_key: "9e849262b2ad4eb249df2853b861f5c1"
server_key_iv: "c9b5e06e070cf102805dd7db4eae8c14"
server_key_version: "encryptionkey@0"
I20260811 10:09:54.113006 5569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:09:54.113853 5589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:54.114024 5569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:54.114127 5569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal
uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a"
format_stamp: "Formatted at 2026-08-11 10:09:54 on dist-test-slave-g350"
server_key: "9e849262b2ad4eb249df2853b861f5c1"
server_key_iv: "c9b5e06e070cf102805dd7db4eae8c14"
server_key_version: "encryptionkey@0"
I20260811 10:09:54.114207 5569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:54.123267 5569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:54.125957 5569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:54.126121 5569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:54.130537 5569 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.60:33419
I20260811 10:09:54.130570 5644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.60:33419 every 8 connection(s)
I20260811 10:09:54.130964 5569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
I20260811 10:09:54.131911 5645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:54.133560 5645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442994, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:54.140280 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5569
I20260811 10:09:54.140456 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786442825609099-23236-0/minicluster-data/master-2/wal/instance
I20260811 10:09:54.140725 23236 external_mini_cluster.cc:1428] Setting key b4aeb8489887649863f50279924bdfeb
I20260811 10:09:54.144979 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.134543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51784 (local address 127.22.177.62:34507)
0811 10:09:54.134630 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.134633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.134917 (+ 284us) server_negotiation.cc:407] Connection header received
0811 10:09:54.135476 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.135478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.135490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.135553 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.136252 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.136787 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.137987 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.138199 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.138282 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.140928 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.140934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.140935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.140942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.143436 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.143897 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.143899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.143899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.143972 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.144271 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.144272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.144272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.144386 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.144469 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.144767 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.144873 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 10:09:54.145220 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.134516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:34507 (local address 127.0.0.1:51784)
0811 10:09:54.134777 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.134823 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:54.135374 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:54.135622 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:54.135626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:54.135946 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:54.136153 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.136158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.136889 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.136890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.137908 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.137911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.138329 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.138330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.138450 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.139074 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:54.139083 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:54.140816 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:54.143536 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.143543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.143545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.143821 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.144055 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.144056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.144056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.144195 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.144463 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:54.144472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:54.144622 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 10:09:54.144781 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 10:09:54.144956 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":113,"threads_started":1}
I20260811 10:09:54.146008 5645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442994, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:09:54.151196 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.141159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.22.177.60:33419)
0811 10:09:54.141334 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.141344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.141436 (+ 92us) server_negotiation.cc:407] Connection header received
0811 10:09:54.141486 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.141488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.141501 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.141570 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.142406 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.143140 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.144277 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.144615 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.144749 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.147700 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.147709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.147709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.147722 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.149695 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.150206 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.150209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.150209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.150295 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.150538 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.150539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.150539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.150705 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.150886 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.150988 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.151074 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260811 10:09:54.153889 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.146322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38029 (local address 127.0.0.1:49186)
0811 10:09:54.146439 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.146449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:54.146491 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:54.146650 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:54.146651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:54.146775 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:54.146824 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.146826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.147706 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.147707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.148394 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.148397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.148804 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.148805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.148884 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.149249 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:54.149253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:54.150681 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:54.152830 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.152831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.152832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.153008 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.153215 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.153216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.153216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.153277 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.153533 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:54.153534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:54.153580 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 10:09:54.153673 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 10:09:54.153793 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 10:09:54.153889 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.146462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.22.177.61:38029)
0811 10:09:54.146517 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.146519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.146528 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:09:54.146555 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.146556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.146569 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.146650 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.146918 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.147605 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.148494 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.148715 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.148805 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.150766 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.150768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.150768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.150776 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.152723 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.153082 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.153083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.153083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.153148 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.153346 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.153346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.153347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.153466 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.153542 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.153703 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.153786 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:09:54.154358 5645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442994, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:09:54.161181 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.152109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.22.177.62:34507)
0811 10:09:54.152157 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.152159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.152167 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:09:54.152190 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.152191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.152201 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.152371 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.152867 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.153727 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.154914 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.155216 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.155275 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.157990 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.157992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.157993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.158001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.159691 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.160258 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.160259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.160260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.160325 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.160560 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.160561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.160561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.160677 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.160838 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.160959 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.161059 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:09:54.162179 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.154643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56424)
0811 10:09:54.154693 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.154703 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:54.154740 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:54.154887 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:54.154888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:54.155045 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:54.155084 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.155085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.155923 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.155924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.156470 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.156472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.156948 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.156949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.157010 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.157371 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:54.157376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:54.158784 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:54.160954 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.160955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.160956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.161174 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.161428 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.161429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.161430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.161503 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.161825 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:54.161826 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:54.161866 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 10:09:54.161951 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:09:54.162063 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 10:09:54.162199 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.154698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.22.177.60:33419)
0811 10:09:54.154734 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.154736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.154744 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:09:54.154786 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.154787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.154798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.154870 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.155166 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.155841 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.156553 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.156862 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.156920 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.158928 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.158930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.158930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.158938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.160870 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.161265 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.161267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.161267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.161340 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.161573 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.161574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.161574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.161746 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.161841 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.161985 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.162113 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":18}
I20260811 10:09:54.164017 5645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: Bootstrap starting.
I20260811 10:09:54.165222 5645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:54.165666 5645 log.cc:824] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:54.166662 5645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: No bootstrap required, opened a new log
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:09:54.168721 5645 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.168865 5645 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:54.168922 5645 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1354fddfb2345bb9dbc4fb4ed977a8a, State: Initialized, Role: FOLLOWER
I20260811 10:09:54.169075 5645 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.169448 5652 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.169510 5645 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:54.169544 5652 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.170991 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.162066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.22.177.61:38029)
0811 10:09:54.162114 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.162117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.162126 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:09:54.162186 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.162187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.162199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.162487 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.163193 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.163716 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.164805 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.165000 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.165053 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.167944 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.167945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.167946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.167953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.169722 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.170081 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.170083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.170083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.170144 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.170389 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.170390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.170391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.170518 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.170741 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.170780 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.170876 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:09:54.171669 5663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:09:54.171850 5663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:09:54.172091 5569 master_runner.cc:186] Error getting master registration for 127.22.177.62:34507: OK, instance_id { permanent_uuid: "372c2505522947638e7355bf1701be72" instance_seqno: 1786442993736179 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:09:54.174109 5569 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:09:54.174165 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:09:54.184423 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.175806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.22.177.62:34507)
0811 10:09:54.175860 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.175862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.175955 (+ 93us) server_negotiation.cc:407] Connection header received
0811 10:09:54.176068 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.176069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.176082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.176115 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.177070 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.177877 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.178963 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.179266 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.179313 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.181209 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.181211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.181211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.181220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.183046 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.183486 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.183487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.183488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.183550 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.183755 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.183756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.183756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.183901 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.183980 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.184215 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.184303 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:09:54.184836 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.175753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.22.177.61:38029)
0811 10:09:54.175855 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.175857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.175964 (+ 107us) server_negotiation.cc:407] Connection header received
0811 10:09:54.176072 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.176073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.176085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.176118 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.177273 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.178084 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.179237 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.179535 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.179577 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.181429 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.181430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.181431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.181439 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.183453 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.183878 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.183879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.183880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.183932 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.184185 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.184186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.184186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.184310 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.184397 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.184633 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.184725 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 10:09:54.185010 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.175578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.22.177.60:33419)
0811 10:09:54.175627 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.175629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.175974 (+ 345us) server_negotiation.cc:407] Connection header received
0811 10:09:54.176072 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.176073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.176084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.176118 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.177267 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.178099 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.179236 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.179534 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.179577 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.181429 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.181430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.181431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.181439 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.183315 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.183825 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.183826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.183827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.183926 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.184177 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.184179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.184179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.184386 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.184468 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.184816 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.184918 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:09:54.185202 5469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } attempt: 3
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:09:54.193336 5560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } attempt: 3
I20260811 10:09:54.193740 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.185438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56450)
0811 10:09:54.185547 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.185558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:54.185618 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:54.186064 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:54.186065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:54.186210 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:54.186252 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.186254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.187142 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.187143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.187685 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.187687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.188182 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.188183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.188234 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.188746 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:54.188752 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:54.190460 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:54.192496 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.192497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.192498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.192694 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.192925 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.192926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.192927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.192996 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.193278 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:54.193279 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:54.193369 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 10:09:54.193448 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 10:09:54.193575 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 10:09:54.194067 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.185902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56450 (local address 127.22.177.60:33419)
0811 10:09:54.185949 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.185951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.185960 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:09:54.185984 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.185985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.185995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.186053 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.186341 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.187047 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.187775 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.188079 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.188153 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.190562 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.190564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.190564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.190572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.192387 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.192771 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.192772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.192773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.192838 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.193067 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.193068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.193068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.193210 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.193281 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.193570 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.193692 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:09:54.196244 5469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: Bootstrap starting.
I20260811 10:09:54.197050 5469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:54.197383 5469 log.cc:824] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: Log is configured to *not* fsync() on all Append() calls
Aug 11 10:09:54 dist-test-slave-g350 krb5kdc[5380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442993, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:09:54.198372 5469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: No bootstrap required, opened a new log
I20260811 10:09:54.200598 5469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.200762 5469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:54.200805 5469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372c2505522947638e7355bf1701be72, State: Initialized, Role: FOLLOWER
I20260811 10:09:54.200961 5469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.201355 5469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:54.201331 5679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.201439 5679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.202246 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.193993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:33419 (local address 127.0.0.1:56460)
0811 10:09:54.194050 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:54.194061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:54.194133 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:54.194265 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:54.194266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:54.194471 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:54.194532 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.194535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.195277 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.195278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.195777 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:54.195778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.196206 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.196207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:54.196267 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.196760 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:54.196766 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:54.198328 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:54.200738 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.200739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.200739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.200983 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.201310 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:54.201311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:54.201311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:54.201384 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:54.201839 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:54.201840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:54.201892 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 10:09:54.201969 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:09:54.202103 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 10:09:54.202652 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:54.193729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.22.177.60:33419)
0811 10:09:54.193993 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:54.193995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:54.194116 (+ 121us) server_negotiation.cc:407] Connection header received
0811 10:09:54.194191 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:54.194192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:54.194203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:54.194262 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:54.194669 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.195188 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.195885 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:54.196112 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:54.196202 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:54.198477 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:54.198479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:54.198479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:54.198488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:54.200625 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.201101 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.201103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.201104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.201213 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:54.201560 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:54.201561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:54.201562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:54.201768 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:54.201846 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:54.202409 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 10:09:54.202513 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":201,"threads_started":1}
I20260811 10:09:54.204476 5560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73: Bootstrap starting.
I20260811 10:09:54.205152 5560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:54.205453 5560 log.cc:824] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:54.206352 5560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73: No bootstrap required, opened a new log
W20260811 10:09:54.206357 5692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:09:54.206405 5692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:09:54.208616 5560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.208760 5560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:54.208814 5560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f76b46f55e841c99afdfb1fefcd0d73, State: Initialized, Role: FOLLOWER
I20260811 10:09:54.208958 5560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.209264 5696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.209383 5560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:54.209417 5696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.206830 5395 master_runner.cc:186] Error getting master registration for 127.22.177.61:38029: OK, instance_id { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" instance_seqno: 1786442993950656 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:09:54.210176 5395 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 10:09:54.211707 5707 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:09:54.211766 5707 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:09:54.213987 5477 master_runner.cc:263] No leader master found from master 1f76b46f55e841c99afdfb1fefcd0d73
I20260811 10:09:54.214038 5477 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 10:09:54.233952 5418 tablet.cc:2369] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:09:54.250317 5696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:09:54.250480 5696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.250780 5696 leader_election.cc:288] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 372c2505522947638e7355bf1701be72 (127.22.177.62:34507), f1354fddfb2345bb9dbc4fb4ed977a8a (127.22.177.60:33419)
I20260811 10:09:54.251454 5618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" is_pre_election: true
I20260811 10:09:54.251629 5618 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f76b46f55e841c99afdfb1fefcd0d73 in term 0.
I20260811 10:09:54.252396 5440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c2505522947638e7355bf1701be72" is_pre_election: true
I20260811 10:09:54.252573 5440 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f76b46f55e841c99afdfb1fefcd0d73 in term 0.
I20260811 10:09:54.252992 5499 leader_election.cc:302] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f76b46f55e841c99afdfb1fefcd0d73, f1354fddfb2345bb9dbc4fb4ed977a8a; no voters:
I20260811 10:09:54.253321 5696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:09:54.253377 5696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:09:54.253400 5696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:54.254936 5696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.255430 5440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c2505522947638e7355bf1701be72"
I20260811 10:09:54.255537 5440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:54.256719 5440 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f76b46f55e841c99afdfb1fefcd0d73 in term 1.
I20260811 10:09:54.257079 5498 leader_election.cc:302] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f76b46f55e841c99afdfb1fefcd0d73, 372c2505522947638e7355bf1701be72; no voters:
I20260811 10:09:54.258813 5618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a"
I20260811 10:09:54.258908 5618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:54.259859 5618 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f76b46f55e841c99afdfb1fefcd0d73 in term 1.
I20260811 10:09:54.260165 5696 leader_election.cc:288] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [CANDIDATE]: Term 1 election: Requested vote from peers 372c2505522947638e7355bf1701be72 (127.22.177.62:34507), f1354fddfb2345bb9dbc4fb4ed977a8a (127.22.177.60:33419)
I20260811 10:09:54.260309 5696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:09:54.260401 5696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [term 1 LEADER]: Becoming Leader. State: Replica: 1f76b46f55e841c99afdfb1fefcd0d73, State: Running, Role: LEADER
I20260811 10:09:54.260543 5696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } }
I20260811 10:09:54.261152 5696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f76b46f55e841c99afdfb1fefcd0d73. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.261312 5696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:54.262171 5711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:09:54.262459 5711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:09:54.264304 5618 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [term 1 FOLLOWER]: Refusing update from remote peer 1f76b46f55e841c99afdfb1fefcd0d73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:09:54.264747 5709 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:09:54.266294 5652 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f76b46f55e841c99afdfb1fefcd0d73. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.266394 5652 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.266950 5440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [term 1 FOLLOWER]: Refusing update from remote peer 1f76b46f55e841c99afdfb1fefcd0d73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:09:54.267320 5709 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:09:54.268682 5679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f76b46f55e841c99afdfb1fefcd0d73. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.268803 5679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.270100 5709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.270195 5709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:54.271739 5679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.271831 5679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.272373 5712 mvcc.cc:205] Tried to move back new op lower bound from 7317270504503480320 to 7317270504492302336. Current Snapshot: MvccSnapshot[applied={T|T < 7317270504503480320}]
I20260811 10:09:54.273221 5715 mvcc.cc:205] Tried to move back new op lower bound from 7317270504503480320 to 7317270504492302336. Current Snapshot: MvccSnapshot[applied={T|T < 7317270504503480320}]
I20260811 10:09:54.273927 5711 catalog_manager.cc:1347] Generated new cluster ID: e6b63fd1eb7a4b9ca25a13425e1251b3
I20260811 10:09:54.273979 5711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:09:54.277904 5696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.278041 5696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:54.278789 5652 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c2505522947638e7355bf1701be72" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34507 } } peers { permanent_uuid: "1f76b46f55e841c99afdfb1fefcd0d73" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38029 } } peers { permanent_uuid: "f1354fddfb2345bb9dbc4fb4ed977a8a" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 33419 } } }
I20260811 10:09:54.278892 5652 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:09:54.307507 5711 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:09:54.308386 5711 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:09:54.325174 5711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f76b46f55e841c99afdfb1fefcd0d73: Generated new TSK 0
I20260811 10:09:54.325520 5711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:09:55.172451 5663 catalog_manager.cc:1259] Loaded cluster ID: e6b63fd1eb7a4b9ca25a13425e1251b3
I20260811 10:09:55.172551 5663 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: loading cluster ID for follower catalog manager: success
I20260811 10:09:55.173532 5663 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: acquiring CA information for follower catalog manager: success
I20260811 10:09:55.173933 5663 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f1354fddfb2345bb9dbc4fb4ed977a8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:09:55.206881 5692 catalog_manager.cc:1259] Loaded cluster ID: e6b63fd1eb7a4b9ca25a13425e1251b3
I20260811 10:09:55.206965 5692 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: loading cluster ID for follower catalog manager: success
I20260811 10:09:55.207868 5692 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: acquiring CA information for follower catalog manager: success
I20260811 10:09:55.208209 5692 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 372c2505522947638e7355bf1701be72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:09:55.384480 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5395
I20260811 10:09:55.393913 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5477
I20260811 10:09:55.403191 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5569
2026-08-11T10:09:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3067 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:09:55 dist-test-slave-g350 krb5kdc[5727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:09:55 dist-test-slave-g350 krb5kdc[5727](info): set up 2 sockets
Aug 11 10:09:55 dist-test-slave-g350 krb5kdc[5727](info): commencing operation
krb5kdc: starting...
W20260811 10:09:56.558686 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:09:56 dist-test-slave-g350 krb5kdc[5727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:09:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:09:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:09:56.702029 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:43011
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:43011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:56.791604 5741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:56.791872 5741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:56.791913 5741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:56.793164 5741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:56.793241 5741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:56.793265 5741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:56.793286 5741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:56.793315 5741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:56.794955 5741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:43011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:43011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:56.795274 5741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:56.795516 5741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:56.801070 5748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:56.801045 5746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:56.801397 5741 server_base.cc:1034] running on GCE node
W20260811 10:09:56.801045 5750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:56.801810 5741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:56.802160 5741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:56.803375 5741 hybrid_clock.cc:648] HybridClock initialized: now 1786442996803360 us; error 47 us; skew 500 ppm
Aug 11 10:09:56 dist-test-slave-g350 krb5kdc[5727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442996, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:56.806476 5741 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:09:56.806985 5741 webserver.cc:460] Webserver started at http://127.22.177.62:34349/ using document root <none> and password file <none>
I20260811 10:09:56.807528 5741 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:56.807614 5741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:56.807781 5741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:56.808998 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "35197ed0152f451f9f969a5e99dfd2f2"
format_stamp: "Formatted at 2026-08-11 10:09:56 on dist-test-slave-g350"
server_key: "2ad16c17245ec84bd9d1eb976a5020a7"
server_key_iv: "f0164475926ef0f95d2b9da6240046f6"
server_key_version: "encryptionkey@0"
I20260811 10:09:56.809453 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "35197ed0152f451f9f969a5e99dfd2f2"
format_stamp: "Formatted at 2026-08-11 10:09:56 on dist-test-slave-g350"
server_key: "2ad16c17245ec84bd9d1eb976a5020a7"
server_key_iv: "f0164475926ef0f95d2b9da6240046f6"
server_key_version: "encryptionkey@0"
I20260811 10:09:56.810981 5741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:56.811909 5758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:56.812101 5741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:56.812214 5741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "35197ed0152f451f9f969a5e99dfd2f2"
format_stamp: "Formatted at 2026-08-11 10:09:56 on dist-test-slave-g350"
server_key: "2ad16c17245ec84bd9d1eb976a5020a7"
server_key_iv: "f0164475926ef0f95d2b9da6240046f6"
server_key_version: "encryptionkey@0"
I20260811 10:09:56.812291 5741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:56.826380 5741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:56.829381 5741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:56.829547 5741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:56.833765 5741 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:43011
I20260811 10:09:56.833822 5816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:43011 every 8 connection(s)
I20260811 10:09:56.834260 5741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:09:56.835295 5817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:56.838457 5817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2: Bootstrap starting.
I20260811 10:09:56.838903 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5741
I20260811 10:09:56.839054 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:09:56.839265 5817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:56.839373 23236 external_mini_cluster.cc:1428] Setting key 00fb463d0e74e261f3fbc1bd407a0a8d
I20260811 10:09:56.839681 5817 log.cc:824] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:56.840996 5817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2: No bootstrap required, opened a new log
I20260811 10:09:56.843056 5817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35197ed0152f451f9f969a5e99dfd2f2" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43011 } }
I20260811 10:09:56.843230 5817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:56.843274 5817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35197ed0152f451f9f969a5e99dfd2f2, State: Initialized, Role: FOLLOWER
I20260811 10:09:56.843441 5817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35197ed0152f451f9f969a5e99dfd2f2" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43011 } }
I20260811 10:09:56.843537 5817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:09:56.843585 5817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:09:56.843633 5817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:56.844343 5817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35197ed0152f451f9f969a5e99dfd2f2" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43011 } }
I20260811 10:09:56.844485 5817 leader_election.cc:302] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35197ed0152f451f9f969a5e99dfd2f2; no voters:
I20260811 10:09:56.844723 5817 leader_election.cc:288] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:09:56.844755 5822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:09:56.845057 5817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:56.845278 5822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [term 1 LEADER]: Becoming Leader. State: Replica: 35197ed0152f451f9f969a5e99dfd2f2, State: Running, Role: LEADER
I20260811 10:09:56.845384 5822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35197ed0152f451f9f969a5e99dfd2f2" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43011 } }
I20260811 10:09:56.846347 5825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35197ed0152f451f9f969a5e99dfd2f2. Latest consensus state: current_term: 1 leader_uuid: "35197ed0152f451f9f969a5e99dfd2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35197ed0152f451f9f969a5e99dfd2f2" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43011 } } }
I20260811 10:09:56.846423 5823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35197ed0152f451f9f969a5e99dfd2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35197ed0152f451f9f969a5e99dfd2f2" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 43011 } } }
I20260811 10:09:56.846504 5825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:56.846513 5823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:56.846784 5834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:09:56.847398 5834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:09:56 dist-test-slave-g350 krb5kdc[5727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:56.850678 5834 catalog_manager.cc:1347] Generated new cluster ID: 0e6a433f31f64f1a879ee296e73d8654
I20260811 10:09:56.850746 5834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:09:56.852429 5820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:56.840004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.22.177.62:43011)
0811 10:09:56.840246 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:56.840255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:56.840323 (+ 68us) server_negotiation.cc:407] Connection header received
0811 10:09:56.841086 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:56.841097 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:56.841164 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:56.841323 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:56.842065 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:56.842984 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:56.844006 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:56.844513 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:56.844623 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:56.848087 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:56.848092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:56.848092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:56.848106 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:56.850473 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:56.850967 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:56.850970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:56.850971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:56.851079 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:56.851467 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:56.851468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:56.851468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:56.851777 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:56.851916 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:56.852074 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 10:09:56.852216 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260811 10:09:56.853750 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:09:56.866108 5834 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:09:56.866983 5834 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:09:56.871451 5834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35197ed0152f451f9f969a5e99dfd2f2: Generated new TSK 0
I20260811 10:09:56.871714 5834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:09:56.908870 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5741
2026-08-11T10:09:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1502 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:09:56 dist-test-slave-g350 krb5kdc[5850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:09:56 dist-test-slave-g350 krb5kdc[5850](info): set up 2 sockets
Aug 11 10:09:56 dist-test-slave-g350 krb5kdc[5850](info): commencing operation
krb5kdc: starting...
W20260811 10:09:58.038496 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:09:58.178483 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40999
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:58.265998 5863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:58.266234 5863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:58.266261 5863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:58.267449 5863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:58.267494 5863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:58.267509 5863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:58.267522 5863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:58.267535 5863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:58.269075 5863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:58.269304 5863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:58.269502 5863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:58.275218 5869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.275218 5871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.275281 5868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:58.275954 5863 server_base.cc:1034] running on GCE node
I20260811 10:09:58.276283 5863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:58.276590 5863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:58.277772 5863 hybrid_clock.cc:648] HybridClock initialized: now 1786442998277752 us; error 52 us; skew 500 ppm
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:58.284180 5863 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:09:58.284834 5863 webserver.cc:460] Webserver started at http://127.22.177.62:35701/ using document root <none> and password file <none>
I20260811 10:09:58.285079 5863 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:58.285146 5863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:58.285305 5863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:58.286530 5863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "5f26e9e1a9b540a1a51d622806130cd9"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "6cc625e7304510e23851ed3534d30051"
server_key_iv: "3bdef321fb258fff51d05f22ddf288df"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.286909 5863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "5f26e9e1a9b540a1a51d622806130cd9"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "6cc625e7304510e23851ed3534d30051"
server_key_iv: "3bdef321fb258fff51d05f22ddf288df"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.288429 5863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:09:58.289431 5878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.289708 5863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:58.289841 5863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "5f26e9e1a9b540a1a51d622806130cd9"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "6cc625e7304510e23851ed3534d30051"
server_key_iv: "3bdef321fb258fff51d05f22ddf288df"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.289945 5863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:58.303305 5863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:58.305955 5863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:58.306135 5863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:58.310740 5863 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40999
I20260811 10:09:58.310793 5939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40999 every 8 connection(s)
I20260811 10:09:58.311206 5863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:09:58.312266 5940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:58.314829 5940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap starting.
I20260811 10:09:58.315294 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5863
I20260811 10:09:58.315465 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:09:58.315560 5940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:58.315722 23236 external_mini_cluster.cc:1428] Setting key 46ec0fcd1a6f3ac8127bc71f1ef92a7b
I20260811 10:09:58.315938 5940 log.cc:824] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:58.316915 5940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: No bootstrap required, opened a new log
I20260811 10:09:58.319490 5940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:58.319682 5940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:58.319720 5940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f26e9e1a9b540a1a51d622806130cd9, State: Initialized, Role: FOLLOWER
I20260811 10:09:58.319890 5940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:58.319988 5940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:09:58.320013 5940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:09:58.320088 5940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:58.320797 5940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:58.320915 5940 leader_election.cc:302] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f26e9e1a9b540a1a51d622806130cd9; no voters:
I20260811 10:09:58.321108 5940 leader_election.cc:288] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:09:58.321216 5946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:09:58.321405 5940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:58.321444 5946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 LEADER]: Becoming Leader. State: Replica: 5f26e9e1a9b540a1a51d622806130cd9, State: Running, Role: LEADER
I20260811 10:09:58.321801 5946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:58.322386 5948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f26e9e1a9b540a1a51d622806130cd9. Latest consensus state: current_term: 1 leader_uuid: "5f26e9e1a9b540a1a51d622806130cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } } }
I20260811 10:09:58.322521 5948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:58.322736 5947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f26e9e1a9b540a1a51d622806130cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } } }
I20260811 10:09:58.322824 5956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:09:58.322865 5947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:58.323362 5956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:58.327550 5956 catalog_manager.cc:1347] Generated new cluster ID: 72ae889b77a042038ba3a7ffbf58f97a
I20260811 10:09:58.327621 5956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:09:58.328778 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.316368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.22.177.62:40999)
0811 10:09:58.316561 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.316571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.316609 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:09:58.317265 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.317275 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.317342 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.317506 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:58.318254 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.319698 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.320636 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.321296 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.321812 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.324711 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:58.324716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:58.324716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:58.324730 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:58.326839 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.327314 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.327318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.327319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.327477 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.327798 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.327799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.327800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.328080 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:58.328268 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.328392 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.328560 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260811 10:09:58.343446 5956 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:09:58.344064 5956 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:09:58.350795 5956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Generated new TSK 0
I20260811 10:09:58.351090 5956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:09:58.390360 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40999
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:58.494217 5968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:58.494443 5968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:58.494469 5968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:58.495704 5968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:58.495756 5968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:58.495784 5968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:09:58.497419 5968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:58.497735 5968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:58.498049 5968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:58.504341 5974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.504421 5975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:58.504375 5968 server_base.cc:1034] running on GCE node
W20260811 10:09:58.504312 5977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:58.504823 5968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:58.505092 5968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:58.506275 5968 hybrid_clock.cc:648] HybridClock initialized: now 1786442998506250 us; error 39 us; skew 500 ppm
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:58.509034 5968 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:09:58.509555 5968 webserver.cc:460] Webserver started at http://127.22.177.1:34307/ using document root <none> and password file <none>
I20260811 10:09:58.509853 5968 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:58.509928 5968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:58.510078 5968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:58.511160 5968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "014a3a59df3548469931bd4765e68aff"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "e2f110fc35d69fcedaa3e23aa2c9d02e"
server_key_iv: "57f7bd1f45b233a27da7251c04e46d66"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.511540 5968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "014a3a59df3548469931bd4765e68aff"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "e2f110fc35d69fcedaa3e23aa2c9d02e"
server_key_iv: "57f7bd1f45b233a27da7251c04e46d66"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.513051 5968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:09:58.513979 5985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.514175 5968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:58.514285 5968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "014a3a59df3548469931bd4765e68aff"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "e2f110fc35d69fcedaa3e23aa2c9d02e"
server_key_iv: "57f7bd1f45b233a27da7251c04e46d66"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.514380 5968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:58.523098 5968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:58.525292 5968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:58.525485 5968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:58.525843 5968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:58.526204 5968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:58.526257 5968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.526297 5968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:58.526325 5968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.533375 5968 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:44881
I20260811 10:09:58.533474 6110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:44881 every 8 connection(s)
I20260811 10:09:58.533840 5968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:09:58.538233 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 5968
I20260811 10:09:58.538380 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:09:58.538596 23236 external_mini_cluster.cc:1428] Setting key c8db3ad61ffcb5e4f089c81088e3fa04
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:58.545017 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.534759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:60547 (local address 127.22.177.62:40999)
0811 10:09:58.534818 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.534821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.535009 (+ 188us) server_negotiation.cc:407] Connection header received
0811 10:09:58.535505 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.535506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.535517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.535585 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:58.536584 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.537095 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.538218 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.538504 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.538601 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.541162 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:58.541164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:58.541164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:58.541174 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:58.543166 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.543742 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.543744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.543744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.543826 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.544155 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.544157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.544157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.544331 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:58.544410 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.544853 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.544942 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 10:09:58.545352 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.534700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:60547)
0811 10:09:58.534876 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:58.534921 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:58.535419 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:58.535638 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:58.535641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:58.536275 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:58.536485 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.536490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.537192 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.537193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.538115 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.538118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.538618 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.538619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.538791 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.539546 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:58.539555 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:58.541005 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:58.543276 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:58.543286 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:58.543289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:58.543603 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:58.543936 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:58.543937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:58.543937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:58.544072 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:58.544447 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:58.544456 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:58.544661 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 10:09:58.544810 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 10:09:58.545023 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260811 10:09:58.546084 6112 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40999
I20260811 10:09:58.546223 6112 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:58.546453 6112 heartbeater.cc:511] Master 127.22.177.62:40999 requested a full tablet report, sending...
I20260811 10:09:58.547120 5899 ts_manager.cc:194] Registered new tserver with Master: 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881)
I20260811 10:09:58.547910 5899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:60547
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:09:58.574703 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40999
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:58.662380 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:58.662633 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:58.662663 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:58.664005 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:58.664053 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:58.664084 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:09:58.665777 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:58.666013 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:58.666239 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:58.672219 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.672310 6127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.672253 6124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:58.672497 6118 server_base.cc:1034] running on GCE node
I20260811 10:09:58.672698 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:58.672935 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:58.674132 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786442998674100 us; error 60 us; skew 500 ppm
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:58.677021 6118 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:09:58.677649 6118 webserver.cc:460] Webserver started at http://127.22.177.2:39183/ using document root <none> and password file <none>
I20260811 10:09:58.677944 6118 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:58.678020 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:58.678180 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:58.679373 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "8c90c49531d34ae890c64021b91e9ae1"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "02c1949c8ac881dd0d338042d0984768"
server_key_iv: "887e4cd781856f9d8b1c6dea2289fc6c"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.679796 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "8c90c49531d34ae890c64021b91e9ae1"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "02c1949c8ac881dd0d338042d0984768"
server_key_iv: "887e4cd781856f9d8b1c6dea2289fc6c"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.681408 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:09:58.682397 6134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.682601 6118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:58.682752 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "8c90c49531d34ae890c64021b91e9ae1"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "02c1949c8ac881dd0d338042d0984768"
server_key_iv: "887e4cd781856f9d8b1c6dea2289fc6c"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.682837 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:58.699916 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:58.702725 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:58.702924 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:58.703207 6118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:58.703564 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:58.703622 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.703677 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:58.703718 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.710484 6118 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:44293
I20260811 10:09:58.710542 6265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:44293 every 8 connection(s)
I20260811 10:09:58.710928 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:09:58.711530 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 6118
I20260811 10:09:58.711645 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:09:58.711874 23236 external_mini_cluster.cc:1428] Setting key 28ebbeb6a0e2abf72719aa68fab26d42
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:09:58.723106 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.711845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:33045 (local address 127.22.177.62:40999)
0811 10:09:58.712148 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.712150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.712267 (+ 117us) server_negotiation.cc:407] Connection header received
0811 10:09:58.712918 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.712920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.712933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.712991 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:58.714115 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.714871 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.716529 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.716877 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.716947 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.719469 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:58.719471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:58.719471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:58.719481 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:58.721323 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.721846 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.721848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.721849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.721914 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.722332 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.722334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.722334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.722501 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:58.722579 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.722886 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.722979 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260811 10:09:58.723423 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.711873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:33045)
0811 10:09:58.712099 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:58.712159 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:58.712815 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:58.713065 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:58.713069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:58.713798 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:58.714020 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.714025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.715000 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.715002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.716388 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.716391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.716962 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.716963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.717089 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.717714 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:58.717723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:58.719371 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:58.721408 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:58.721415 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:58.721418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:58.721734 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:58.722025 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:58.722026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:58.722026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:58.722166 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:58.722582 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:58.722589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:58.722738 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 10:09:58.722825 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 10:09:58.723055 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260811 10:09:58.723955 6266 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40999
I20260811 10:09:58.724074 6266 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:58.724280 6266 heartbeater.cc:511] Master 127.22.177.62:40999 requested a full tablet report, sending...
I20260811 10:09:58.724727 5899 ts_manager.cc:194] Registered new tserver with Master: 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
I20260811 10:09:58.725162 5899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:33045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:09:58.745332 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40999
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:58.832159 6271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:58.832403 6271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:58.832429 6271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:58.833705 6271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:58.833755 6271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:58.833783 6271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:09:58.835368 6271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:58.835598 6271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:58.835796 6271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:58.841477 6277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.841581 6280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:58.841475 6278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:58.841734 6271 server_base.cc:1034] running on GCE node
I20260811 10:09:58.842000 6271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:58.842285 6271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:58.843459 6271 hybrid_clock.cc:648] HybridClock initialized: now 1786442998843442 us; error 37 us; skew 500 ppm
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:58.846386 6271 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:09:58.846911 6271 webserver.cc:460] Webserver started at http://127.22.177.3:43461/ using document root <none> and password file <none>
I20260811 10:09:58.847213 6271 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:58.847287 6271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:58.847434 6271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:09:58.848542 6271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "af4a47638bc24725accd08c6f955a5f0"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "49e022da9607929b254e985baa4f3d8c"
server_key_iv: "6bbe9a5a6a82271fc581c108a3bbe973"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.848930 6271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "af4a47638bc24725accd08c6f955a5f0"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "49e022da9607929b254e985baa4f3d8c"
server_key_iv: "6bbe9a5a6a82271fc581c108a3bbe973"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.850615 6271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:09:58.851555 6287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.851728 6271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:58.851856 6271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "af4a47638bc24725accd08c6f955a5f0"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "49e022da9607929b254e985baa4f3d8c"
server_key_iv: "6bbe9a5a6a82271fc581c108a3bbe973"
server_key_version: "encryptionkey@0"
I20260811 10:09:58.851917 6271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:58.874791 6271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:58.877358 6271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:58.877501 6271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:58.877803 6271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:58.878216 6271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:58.878252 6271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.878283 6271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:58.878316 6271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:58.884683 6271 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:46551
I20260811 10:09:58.884735 6415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:46551 every 8 connection(s)
I20260811 10:09:58.885152 6271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:09:58.892241 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 6271
I20260811 10:09:58.892366 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:09:58.892561 23236 external_mini_cluster.cc:1428] Setting key 63ca08f0bc2db8b10f64b271806517a6
Aug 11 10:09:58 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442998, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:58.896786 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.886128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52775 (local address 127.22.177.62:40999)
0811 10:09:58.886347 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.886350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.886583 (+ 233us) server_negotiation.cc:407] Connection header received
0811 10:09:58.887386 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.887388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.887423 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.887484 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:58.889027 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.889706 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.890858 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.891086 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.891190 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.894045 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:58.894047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:58.894047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:58.894055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:58.895418 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.895841 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.895842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.895842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.895925 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.896165 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.896166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.896166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.896309 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:58.896372 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.896623 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.896706 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260811 10:09:58.896992 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.886083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:52775)
0811 10:09:58.886445 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:58.886494 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:58.887258 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:58.887598 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:58.887603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:58.888514 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:58.888909 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.888918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.889820 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.889821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.890738 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.890740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.891226 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.891228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.891382 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.891965 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:58.891975 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:58.893960 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:58.895496 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:58.895503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:58.895505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:58.895749 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:58.895998 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:58.895999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:58.895999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:58.896106 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:58.896369 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:58.896376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:58.896503 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 10:09:58.896590 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 10:09:58.896720 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":98,"threads_started":1}
I20260811 10:09:58.897718 6416 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40999
I20260811 10:09:58.897838 6416 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:58.898082 6416 heartbeater.cc:511] Master 127.22.177.62:40999 requested a full tablet report, sending...
I20260811 10:09:58.898608 5899 ts_manager.cc:194] Registered new tserver with Master: af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551)
I20260811 10:09:58.899088 5899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:52775
I20260811 10:09:58.905036 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:09:58.913993 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.906766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.22.177.62:40999)
0811 10:09:58.906856 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.906858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.906953 (+ 95us) server_negotiation.cc:407] Connection header received
0811 10:09:58.907010 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.907010 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.907029 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.907073 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:58.907663 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.908157 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.908935 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.909125 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.909194 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.910839 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:58.910840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:58.910841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:58.910849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:58.912503 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.912911 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.912912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.912913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.912980 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:58.913222 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:58.913223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:58.913223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:58.913402 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:58.913489 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.913773 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.913893 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 10:09:58.916013 5899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:09:58.917745 5899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:09:58.924666 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.920219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:46551 (local address 127.0.0.1:34002)
0811 10:09:58.920433 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:58.920480 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:58.920554 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:58.920791 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:58.920795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:58.920797 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:58.920966 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.920968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.922420 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.922421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.923426 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.923429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.924323 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.924324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.924413 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.924427 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:09:58.924506 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 10:09:58.924569 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":62,"threads_started":1}
I20260811 10:09:58.924706 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.920394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.22.177.3:46551)
0811 10:09:58.920617 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.920623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.920633 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:09:58.920672 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.920674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.920688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.920786 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:58.921060 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.922289 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.923527 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.924154 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.924225 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.924285 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.924551 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.924601 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":105,"threads_started":1}
I20260811 10:09:58.925161 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.920461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:44881 (local address 127.0.0.1:41056)
0811 10:09:58.920895 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:58.920906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:58.920967 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:58.921264 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:58.921265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:58.921267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:58.921314 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.921316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.922348 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.922349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.923835 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.923839 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.924887 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.924888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.924939 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.924943 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:09:58.925028 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:09:58.925076 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":117,"thread_start_us":76,"threads_started":1}
I20260811 10:09:58.925171 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.920724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.22.177.1:44881)
0811 10:09:58.920920 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.920924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.920959 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:09:58.921051 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.921053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.921071 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.921162 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:58.921414 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.922229 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.923965 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.924698 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.924782 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.924840 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.925050 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.925092 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":115,"threads_started":1}
I20260811 10:09:58.925388 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.920222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44293 (local address 127.0.0.1:51112)
0811 10:09:58.920584 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:58.920598 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:58.920661 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:58.921296 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:58.921297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:58.921299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:58.921353 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.921354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.922712 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.922713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.923845 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:58.923847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.925156 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.925157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:58.925232 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.925237 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:09:58.925267 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 10:09:58.925323 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":121,"threads_started":1}
I20260811 10:09:58.925563 6343 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec64a4d404847d28f5bb188cbc7417f (DEFAULT_TABLE table=test-table [id=edf3ebf931004239834fbc747e6715c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:58.926016 6343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec64a4d404847d28f5bb188cbc7417f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:58.925979 6037 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec64a4d404847d28f5bb188cbc7417f (DEFAULT_TABLE table=test-table [id=edf3ebf931004239834fbc747e6715c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:58.926262 6037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec64a4d404847d28f5bb188cbc7417f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:58.928750 6435 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Bootstrap starting.
I20260811 10:09:58.929095 6439 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Bootstrap starting.
I20260811 10:09:58.929715 6435 tablet_bootstrap.cc:654] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:58.929754 6439 tablet_bootstrap.cc:654] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:58.929752 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:58.920354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.22.177.2:44293)
0811 10:09:58.921112 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:58.921116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:58.921125 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:09:58.921166 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:58.921168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:58.921183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:58.921293 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:58.921452 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.922578 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.923941 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:58.924680 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:58.929432 (+ 4752us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:58.929528 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:58.929561 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 10:09:58.929655 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":85,"threads_started":1}
I20260811 10:09:58.930377 6435 log.cc:824] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:58.930377 6439 log.cc:824] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:58.930397 6188 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec64a4d404847d28f5bb188cbc7417f (DEFAULT_TABLE table=test-table [id=edf3ebf931004239834fbc747e6715c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:58.930763 6188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec64a4d404847d28f5bb188cbc7417f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:58.931236 6439 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: No bootstrap required, opened a new log
I20260811 10:09:58.931329 6439 ts_tablet_manager.cc:1397] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:58.931455 6435 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: No bootstrap required, opened a new log
I20260811 10:09:58.931537 6435 ts_tablet_manager.cc:1397] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 10:09:58.932824 6439 raft_consensus.cc:348] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:58.933004 6439 raft_consensus.cc:374] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:58.933046 6439 raft_consensus.cc:729] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a3a59df3548469931bd4765e68aff, State: Initialized, Role: FOLLOWER
I20260811 10:09:58.933169 6439 consensus_queue.cc:260] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:58.933511 6439 ts_tablet_manager.cc:1428] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:09:58.933511 6112 heartbeater.cc:503] Master 127.22.177.62:40999 was elected leader, sending a full tablet report...
I20260811 10:09:58.933727 6435 raft_consensus.cc:348] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:58.933909 6435 raft_consensus.cc:374] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:58.934026 6435 raft_consensus.cc:729] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4a47638bc24725accd08c6f955a5f0, State: Initialized, Role: FOLLOWER
I20260811 10:09:58.934185 6435 consensus_queue.cc:260] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:58.934505 6435 ts_tablet_manager.cc:1428] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 10:09:58.934899 6416 heartbeater.cc:503] Master 127.22.177.62:40999 was elected leader, sending a full tablet report...
I20260811 10:09:58.935005 6444 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Bootstrap starting.
I20260811 10:09:58.935647 6444 tablet_bootstrap.cc:654] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:58.936218 6444 log.cc:824] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:58.936940 6444 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: No bootstrap required, opened a new log
I20260811 10:09:58.937026 6444 ts_tablet_manager.cc:1397] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:58.938621 6444 raft_consensus.cc:348] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:58.938779 6444 raft_consensus.cc:374] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:58.938838 6444 raft_consensus.cc:729] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c90c49531d34ae890c64021b91e9ae1, State: Initialized, Role: FOLLOWER
I20260811 10:09:58.938959 6444 consensus_queue.cc:260] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:58.939254 6444 ts_tablet_manager.cc:1428] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:58.939282 6266 heartbeater.cc:503] Master 127.22.177.62:40999 was elected leader, sending a full tablet report...
W20260811 10:09:58.961689 6267 tablet.cc:2369] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:09:59.034637 6113 tablet.cc:2369] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:09:59.101071 6446 raft_consensus.cc:482] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:09:59.101208 6446 raft_consensus.cc:504] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.101761 6446 leader_election.cc:288] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881)
I20260811 10:09:59.105135 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.102084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:32875 (local address 127.22.177.3:46551)
0811 10:09:59.102149 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.102150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.102285 (+ 135us) server_negotiation.cc:407] Connection header received
0811 10:09:59.102406 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.102407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.102417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.102475 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:59.102644 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.103109 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.104353 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.104757 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.104845 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.104857 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.104997 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.105044 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:09:59.105126 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.101973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:46551 (local address 127.22.177.2:32875)
0811 10:09:59.102201 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.102213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.102279 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.102484 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.102485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.102487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:59.102569 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.102570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.103218 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.103219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.104270 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.104273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.104847 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.104848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.104920 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.104924 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.104974 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.105025 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":21,"thread_start_us":118,"threads_started":1}
I20260811 10:09:59.105165 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.102116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:43563 (local address 127.22.177.1:44881)
0811 10:09:59.102170 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.102172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.102184 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:09:59.102219 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.102220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.102319 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.102378 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:59.102574 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.103089 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.104416 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.104763 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.104849 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.104858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.105045 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.105091 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:09:59.105136 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.101973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:44881 (local address 127.22.177.2:43563)
0811 10:09:59.102049 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.102063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.102134 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.102404 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.102405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.102407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:59.102496 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.102499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.103184 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.103185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.104323 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.104326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.104867 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.104868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.104951 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.104954 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.105029 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.105067 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":74}
I20260811 10:09:59.105772 6366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec64a4d404847d28f5bb188cbc7417f" candidate_uuid: "8c90c49531d34ae890c64021b91e9ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4a47638bc24725accd08c6f955a5f0" is_pre_election: true
I20260811 10:09:59.105808 6060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec64a4d404847d28f5bb188cbc7417f" candidate_uuid: "8c90c49531d34ae890c64021b91e9ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "014a3a59df3548469931bd4765e68aff" is_pre_election: true
I20260811 10:09:59.105940 6366 raft_consensus.cc:2454] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c90c49531d34ae890c64021b91e9ae1 in term 0.
I20260811 10:09:59.105968 6060 raft_consensus.cc:2454] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c90c49531d34ae890c64021b91e9ae1 in term 0.
I20260811 10:09:59.106177 6137 leader_election.cc:302] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0; no voters:
I20260811 10:09:59.106335 6446 raft_consensus.cc:2781] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:09:59.106395 6446 raft_consensus.cc:482] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:09:59.106424 6446 raft_consensus.cc:3037] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:59.107261 6446 raft_consensus.cc:504] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.107419 6446 leader_election.cc:288] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [CANDIDATE]: Term 1 election: Requested vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881)
I20260811 10:09:59.107612 6060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec64a4d404847d28f5bb188cbc7417f" candidate_uuid: "8c90c49531d34ae890c64021b91e9ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "014a3a59df3548469931bd4765e68aff"
I20260811 10:09:59.107604 6366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec64a4d404847d28f5bb188cbc7417f" candidate_uuid: "8c90c49531d34ae890c64021b91e9ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4a47638bc24725accd08c6f955a5f0"
I20260811 10:09:59.107687 6060 raft_consensus.cc:3037] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:59.107726 6366 raft_consensus.cc:3037] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:09:59.108392 6060 raft_consensus.cc:2454] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c90c49531d34ae890c64021b91e9ae1 in term 1.
I20260811 10:09:59.108516 6366 raft_consensus.cc:2454] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c90c49531d34ae890c64021b91e9ae1 in term 1.
I20260811 10:09:59.108583 6136 leader_election.cc:302] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 014a3a59df3548469931bd4765e68aff, 8c90c49531d34ae890c64021b91e9ae1; no voters:
I20260811 10:09:59.108783 6446 raft_consensus.cc:2781] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:09:59.108920 6446 raft_consensus.cc:686] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 1 LEADER]: Becoming Leader. State: Replica: 8c90c49531d34ae890c64021b91e9ae1, State: Running, Role: LEADER
I20260811 10:09:59.109033 6446 consensus_queue.cc:237] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.109930 5899 catalog_manager.cc:5495] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2). New cstate: current_term: 1 leader_uuid: "8c90c49531d34ae890c64021b91e9ae1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } health_report { overall_health: UNKNOWN } } }
W20260811 10:09:59.135970 6417 tablet.cc:2369] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:09:59.154438 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.151634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.22.177.2:44293)
0811 10:09:59.151718 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.151721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.151731 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:09:59.151793 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.151794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.151816 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.151917 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:09:59.152074 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.152605 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.153449 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.153733 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.153812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.154001 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:09:59.154126 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:09:59.154232 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.154260 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.154284 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 10:09:59.156613 6366 raft_consensus.cc:1261] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Refusing update from remote peer 8c90c49531d34ae890c64021b91e9ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:09:59.156635 6060 raft_consensus.cc:1261] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Refusing update from remote peer 8c90c49531d34ae890c64021b91e9ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:09:59.157021 6448 consensus_queue.cc:1035] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:09:59.157171 6446 consensus_queue.cc:1035] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:09:59.160689 6452 mvcc.cc:205] Tried to move back new op lower bound from 7317270524542271488 to 7317270524351098880. Current Snapshot: MvccSnapshot[applied={T|T < 7317270524542271488}]
I20260811 10:09:59.161574 6453 mvcc.cc:205] Tried to move back new op lower bound from 7317270524542271488 to 7317270524351098880. Current Snapshot: MvccSnapshot[applied={T|T < 7317270524542271488 or (T in {7317270524542271488})}]
I20260811 10:09:59.162880 5899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:09:59.163000 5899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:09:59.163978 5899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Sending DeleteTablet for 3 replicas of tablet 7ec64a4d404847d28f5bb188cbc7417f
I20260811 10:09:59.164377 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5968
I20260811 10:09:59.164368 6343 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec64a4d404847d28f5bb188cbc7417f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:09:59 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:34002
I20260811 10:09:59.165544 6188 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec64a4d404847d28f5bb188cbc7417f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:09:59 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:51112
I20260811 10:09:59.166289 6463 tablet_replica.cc:330] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: stopping tablet replica
I20260811 10:09:59.166399 6463 raft_consensus.cc:2229] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:09:59.166507 6463 raft_consensus.cc:2258] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:09:59.167176 6463 ts_tablet_manager.cc:1905] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:09:59.168447 6462 tablet_replica.cc:330] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: stopping tablet replica
I20260811 10:09:59.168608 6462 raft_consensus.cc:2229] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:09:59.168820 6462 raft_consensus.cc:2258] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:09:59.169332 6463 ts_tablet_manager.cc:1918] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:09:59.169406 6463 log.cc:1196] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/7ec64a4d404847d28f5bb188cbc7417f
I20260811 10:09:59.169632 6462 ts_tablet_manager.cc:1905] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:09:59.169726 6463 ts_tablet_manager.cc:1939] T 7ec64a4d404847d28f5bb188cbc7417f P 8c90c49531d34ae890c64021b91e9ae1: Deleting consensus metadata
I20260811 10:09:59.170218 5880 catalog_manager.cc:4841] TS 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): tablet 7ec64a4d404847d28f5bb188cbc7417f (table test-table [id=edf3ebf931004239834fbc747e6715c5]) successfully deleted
I20260811 10:09:59.172147 6462 ts_tablet_manager.cc:1918] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:09:59.172351 6462 log.cc:1196] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/7ec64a4d404847d28f5bb188cbc7417f
I20260811 10:09:59.172730 6462 ts_tablet_manager.cc:1939] T 7ec64a4d404847d28f5bb188cbc7417f P af4a47638bc24725accd08c6f955a5f0: Deleting consensus metadata
I20260811 10:09:59.173369 5881 catalog_manager.cc:4841] TS af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): tablet 7ec64a4d404847d28f5bb188cbc7417f (table test-table [id=edf3ebf931004239834fbc747e6715c5]) successfully deleted
W20260811 10:09:59.180534 5880 connection.cc:535] client connection to 127.22.177.1:44881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 10:09:59.180680 5880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:09:59.180779 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 6118
I20260811 10:09:59.182204 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.181117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:44881 (local address 127.0.0.1:41060)
0811 10:09:59.182107 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.182167 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":956}
W20260811 10:09:59.183079 5880 catalog_manager.cc:4584] TS 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ec64a4d404847d28f5bb188cbc7417f: Network error: Client connection negotiation failed: client connection to 127.22.177.1:44881: connect: Connection refused (error 111)
I20260811 10:09:59.188462 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 6271
I20260811 10:09:59.196368 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 5863
I20260811 10:09:59.204568 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40999
--webserver_interface=127.22.177.62
--webserver_port=35701
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:09:59.292052 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:59.292313 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:59.292354 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:59.293545 6465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:09:59.293648 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:59.293677 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:59.293706 6465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:09:59.293738 6465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:09:59.295257 6465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=35701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:59.295547 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:59.295850 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:59.307240 6476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.307262 6474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:59.307312 6465 server_base.cc:1034] running on GCE node
W20260811 10:09:59.307317 6473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:59.307796 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:59.308154 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:59.309319 6465 hybrid_clock.cc:648] HybridClock initialized: now 1786442999309281 us; error 49 us; skew 500 ppm
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:59.312108 6465 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:09:59.312626 6465 webserver.cc:460] Webserver started at http://127.22.177.62:35701/ using document root <none> and password file <none>
I20260811 10:09:59.312896 6465 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:59.312978 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:59.314543 6465 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:59.315299 6483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.315549 6465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:09:59.315646 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "5f26e9e1a9b540a1a51d622806130cd9"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "6cc625e7304510e23851ed3534d30051"
server_key_iv: "3bdef321fb258fff51d05f22ddf288df"
server_key_version: "encryptionkey@0"
I20260811 10:09:59.316083 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:59.330925 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:59.333387 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:59.333518 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:59.337803 6465 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40999
I20260811 10:09:59.337855 6546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40999 every 8 connection(s)
I20260811 10:09:59.338299 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:09:59.340122 6547 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:09:59.340862 6547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap starting.
I20260811 10:09:59.341868 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 6465
I20260811 10:09:59.342219 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:44881
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:40999
--webserver_port=34307
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:09:59.350644 6547 log.cc:824] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:59.352818 6547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:09:59.353161 6547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap complete.
I20260811 10:09:59.355794 6547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:59.356112 6547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f26e9e1a9b540a1a51d622806130cd9, State: Initialized, Role: FOLLOWER
I20260811 10:09:59.356324 6547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:59.356422 6547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:09:59.356492 6547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:09:59.356546 6547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:09:59.357811 6547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:59.357947 6547 leader_election.cc:302] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f26e9e1a9b540a1a51d622806130cd9; no voters:
I20260811 10:09:59.358163 6547 leader_election.cc:288] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:09:59.358212 6551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:09:59.358346 6551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 LEADER]: Becoming Leader. State: Replica: 5f26e9e1a9b540a1a51d622806130cd9, State: Running, Role: LEADER
I20260811 10:09:59.358461 6547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:09:59.358439 6551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:09:59.358731 6551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f26e9e1a9b540a1a51d622806130cd9. Latest consensus state: current_term: 2 leader_uuid: "5f26e9e1a9b540a1a51d622806130cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } } }
I20260811 10:09:59.358814 6551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:59.358736 6552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5f26e9e1a9b540a1a51d622806130cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } } }
I20260811 10:09:59.358873 6552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:09:59.359015 6555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:09:59.360209 6555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:09:59.360337 6555 catalog_manager.cc:1259] Loaded cluster ID: 72ae889b77a042038ba3a7ffbf58f97a
I20260811 10:09:59.360383 6555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:09:59.360816 6555 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 10:09:59.363729 6555 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 10:09:59.366641 6555 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:09:59.366878 6555 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Loaded TSK: 0
I20260811 10:09:59.367117 6555 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:09:59.443619 6549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:59.443872 6549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:59.443917 6549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:59.445119 6549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:59.445196 6549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:59.445248 6549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:09:59.446862 6549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:44881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=34307
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:59.447144 6549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:59.447391 6549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:59.452883 6579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.452891 6577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.452891 6576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:59.453398 6549 server_base.cc:1034] running on GCE node
I20260811 10:09:59.453650 6549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:59.453926 6549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:59.455122 6549 hybrid_clock.cc:648] HybridClock initialized: now 1786442999455096 us; error 33 us; skew 500 ppm
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:59.458233 6549 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:09:59.458772 6549 webserver.cc:460] Webserver started at http://127.22.177.1:34307/ using document root <none> and password file <none>
I20260811 10:09:59.459149 6549 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:59.459218 6549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:59.460822 6549 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:59.461732 6588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.461937 6549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:09:59.462059 6549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "014a3a59df3548469931bd4765e68aff"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "e2f110fc35d69fcedaa3e23aa2c9d02e"
server_key_iv: "57f7bd1f45b233a27da7251c04e46d66"
server_key_version: "encryptionkey@0"
I20260811 10:09:59.462563 6549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:59.485069 6549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:59.487697 6549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:59.487886 6549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:59.488149 6549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:59.488623 6596 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:09:59.490670 6549 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:09:59.490746 6549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:09:59.490789 6549 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:09:59.491462 6549 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:09:59.491515 6549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:09:59.491577 6596 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Bootstrap starting.
I20260811 10:09:59.499190 6549 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:44881
I20260811 10:09:59.499423 6715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:44881 every 8 connection(s)
I20260811 10:09:59.499677 6549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:09:59.503407 6596 log.cc:824] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:59.506532 6596 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:09:59.506889 6596 tablet_bootstrap.cc:492] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Bootstrap complete.
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:59.508241 6596 ts_tablet_manager.cc:1397] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.005s
I20260811 10:09:59.509754 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 6549
I20260811 10:09:59.509614 6596 raft_consensus.cc:348] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.509925 6596 raft_consensus.cc:729] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a3a59df3548469931bd4765e68aff, State: Initialized, Role: FOLLOWER
I20260811 10:09:59.510052 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:44293
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:40999
--webserver_port=39183
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:09:59.510080 6596 consensus_queue.cc:260] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.510367 6596 ts_tablet_manager.cc:1428] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260811 10:09:59.513469 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.500433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:38807)
0811 10:09:59.500657 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.500710 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.501257 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.501748 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.501752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.502054 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:59.502253 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.502259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.503530 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.503531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.504517 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.504520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.505178 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.505180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.505317 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.505938 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:59.505947 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:59.507544 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:59.509531 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.509540 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.509542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.510382 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.512083 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.512084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.512084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.512226 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.512784 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:59.512794 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:59.512955 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.513068 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.513264 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":111,"threads_started":1}
I20260811 10:09:59.513469 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.500572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:38807 (local address 127.22.177.62:40999)
0811 10:09:59.500814 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.500822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.500865 (+ 43us) server_negotiation.cc:407] Connection header received
0811 10:09:59.501491 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.501496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.501541 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.501714 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:59.502382 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.503396 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.504613 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.505013 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.505139 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.507661 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:59.507665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:59.507665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:59.507677 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:59.509407 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.511834 (+ 2427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.511838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.511839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.511988 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.512310 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.512311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.512311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.512641 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:59.512704 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.513140 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.513297 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
I20260811 10:09:59.514403 6716 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40999
I20260811 10:09:59.514500 6716 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:59.514674 6716 heartbeater.cc:511] Master 127.22.177.62:40999 requested a full tablet report, sending...
I20260811 10:09:59.515410 6506 ts_manager.cc:194] Registered new tserver with Master: 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881)
I20260811 10:09:59.516618 6506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:38807
I20260811 10:09:59.517627 6716 heartbeater.cc:503] Master 127.22.177.62:40999 was elected leader, sending a full tablet report...
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:09:59.525700 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.516020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:44881 (local address 127.0.0.1:41068)
0811 10:09:59.516765 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.516821 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.516920 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.517170 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.517176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.517508 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:59.517565 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.517567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.518921 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.518922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.519697 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.519700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.520407 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.520409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.520483 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.521035 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:59.521040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:59.522446 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:59.524521 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.524528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.524530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.524738 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.524959 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.524959 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.524959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.525019 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.525344 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:59.525351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:59.525406 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.525476 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.525575 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":114,"threads_started":1}
I20260811 10:09:59.525749 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.516308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.22.177.1:44881)
0811 10:09:59.516467 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.516478 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.516912 (+ 434us) server_negotiation.cc:407] Connection header received
0811 10:09:59.517056 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.517059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.517079 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.517149 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:59.517680 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.518839 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.519792 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.520303 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.520390 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.522536 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:59.522545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:59.522545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:59.522553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:59.524430 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.524823 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.524826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.524827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.524894 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.525096 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.525097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.525098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.525275 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:59.525359 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.525527 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.525647 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260811 10:09:59.526144 6642 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec64a4d404847d28f5bb188cbc7417f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:09:59 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41068
I20260811 10:09:59.526696 6726 tablet_replica.cc:330] stopping tablet replica
I20260811 10:09:59.526815 6726 raft_consensus.cc:2229] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:09:59.526891 6726 raft_consensus.cc:2258] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:09:59.527184 6726 ts_tablet_manager.cc:1905] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:09:59.531456 6726 ts_tablet_manager.cc:1918] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:09:59.531540 6726 log.cc:1196] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/7ec64a4d404847d28f5bb188cbc7417f
I20260811 10:09:59.531864 6726 ts_tablet_manager.cc:1939] T 7ec64a4d404847d28f5bb188cbc7417f P 014a3a59df3548469931bd4765e68aff: Deleting consensus metadata
I20260811 10:09:59.532485 6485 catalog_manager.cc:4841] TS 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881): tablet 7ec64a4d404847d28f5bb188cbc7417f (table test-table [id=edf3ebf931004239834fbc747e6715c5]) successfully deleted
W20260811 10:09:59.599746 6723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:59.599987 6723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:59.600026 6723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:59.601198 6723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:59.601263 6723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:59.601308 6723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:09:59.602844 6723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:44293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=39183
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:59.603129 6723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:59.603368 6723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:59.608974 6731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.609083 6735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.609056 6732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:59.609364 6723 server_base.cc:1034] running on GCE node
I20260811 10:09:59.609619 6723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:59.609891 6723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:59.611035 6723 hybrid_clock.cc:648] HybridClock initialized: now 1786442999611017 us; error 36 us; skew 500 ppm
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:59.613759 6723 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:09:59.614244 6723 webserver.cc:460] Webserver started at http://127.22.177.2:39183/ using document root <none> and password file <none>
I20260811 10:09:59.614497 6723 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:59.614583 6723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:59.615994 6723 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:09:59.616681 6742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.617072 6723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:59.617252 6723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "8c90c49531d34ae890c64021b91e9ae1"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "02c1949c8ac881dd0d338042d0984768"
server_key_iv: "887e4cd781856f9d8b1c6dea2289fc6c"
server_key_version: "encryptionkey@0"
I20260811 10:09:59.617681 6723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:59.631944 6723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:59.634409 6723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:59.634586 6723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:59.634845 6723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:59.635157 6723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:59.635210 6723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.635247 6723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:59.635291 6723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.641489 6723 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:44293
I20260811 10:09:59.641537 6870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:44293 every 8 connection(s)
I20260811 10:09:59.641932 6723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:09:59.647305 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 6723
I20260811 10:09:59.647676 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:46551
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:40999
--webserver_port=43461
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:59.653164 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.642683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:51871 (local address 127.22.177.62:40999)
0811 10:09:59.642780 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.642782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.643000 (+ 218us) server_negotiation.cc:407] Connection header received
0811 10:09:59.643653 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.643654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.643666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.643732 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:59.644823 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.645345 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.646530 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.646769 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.646832 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.649681 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:59.649683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:59.649683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:59.649694 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:59.651381 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.651956 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.651957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.651957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.652029 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.652405 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.652407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.652407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.652535 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:59.652615 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.652997 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.653094 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:09:59.653505 6873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.642655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:51871)
0811 10:09:59.642857 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.642904 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.643536 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.643808 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.643811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.644499 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:59.644708 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.644714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.645460 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.645461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.646432 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.646434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.646850 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.646850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.646953 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.647606 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:59.647617 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:59.649519 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:59.651491 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.651502 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.651505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.651859 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.652102 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.652103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.652104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.652272 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.652616 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:59.652624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:59.652824 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.652959 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.653184 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":120,"threads_started":1}
I20260811 10:09:59.654275 6871 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40999
I20260811 10:09:59.654398 6871 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:59.654635 6871 heartbeater.cc:511] Master 127.22.177.62:40999 requested a full tablet report, sending...
I20260811 10:09:59.655146 6506 ts_manager.cc:194] Registered new tserver with Master: 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
I20260811 10:09:59.655745 6506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:51871
W20260811 10:09:59.735586 6874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:09:59.735849 6874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:09:59.735888 6874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:09:59.737071 6874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:09:59.737113 6874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:09:59.737140 6874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:09:59.738729 6874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:46551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=43461
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:09:59.738996 6874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:09:59.739236 6874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:09:59.744733 6883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.744758 6881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:09:59.744745 6880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:09:59.744776 6874 server_base.cc:1034] running on GCE node
I20260811 10:09:59.745174 6874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:09:59.745440 6874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:09:59.746608 6874 hybrid_clock.cc:648] HybridClock initialized: now 1786442999746580 us; error 54 us; skew 500 ppm
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:09:59.749197 6874 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:09:59.749724 6874 webserver.cc:460] Webserver started at http://127.22.177.3:43461/ using document root <none> and password file <none>
I20260811 10:09:59.750010 6874 fs_manager.cc:359] Metadata directory not provided
I20260811 10:09:59.750094 6874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:09:59.751508 6874 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:59.752177 6890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.752511 6874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:09:59.752653 6874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "af4a47638bc24725accd08c6f955a5f0"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "49e022da9607929b254e985baa4f3d8c"
server_key_iv: "6bbe9a5a6a82271fc581c108a3bbe973"
server_key_version: "encryptionkey@0"
I20260811 10:09:59.753057 6874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:09:59.776136 6874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:09:59.778725 6874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:09:59.778934 6874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:09:59.779222 6874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:09:59.779590 6874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:09:59.779650 6874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.779697 6874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:09:59.779732 6874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:09:59.786286 6874 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:46551
I20260811 10:09:59.786346 7014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:46551 every 8 connection(s)
I20260811 10:09:59.786689 6874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:09:59 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786442999, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:09:59.795148 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 6874
I20260811 10:09:59.796335 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.787446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:59895 (local address 127.22.177.62:40999)
0811 10:09:59.787515 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.787517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.787712 (+ 195us) server_negotiation.cc:407] Connection header received
0811 10:09:59.788285 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.788287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.788304 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.788370 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:59.789306 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.789874 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.790909 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.791130 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.791202 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.793402 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:59.793404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:59.793404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:59.793411 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:59.794861 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.795282 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.795283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.795283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.795345 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.795605 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.795606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.795607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.795765 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:59.795854 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.796203 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.796265 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:09:59.796650 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.787445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:59895)
0811 10:09:59.787589 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.787625 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.788109 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.788426 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.788429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.788999 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:09:59.789195 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.789200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.789949 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.789950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.790817 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.790820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.791278 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.791279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.791383 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.791887 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:09:59.791896 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:09:59.793312 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:09:59.794933 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.794940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.794941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.795168 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.795421 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:09:59.795422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:09:59.795422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:09:59.795527 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:09:59.795855 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:09:59.795861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:09:59.796060 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.796189 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.796371 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260811 10:09:59.797150 7016 heartbeater.cc:347] Connected to a master server at 127.22.177.62:40999
I20260811 10:09:59.797259 7016 heartbeater.cc:464] Registering TS with master...
I20260811 10:09:59.797444 7016 heartbeater.cc:511] Master 127.22.177.62:40999 requested a full tablet report, sending...
I20260811 10:09:59.797880 6506 ts_manager.cc:194] Registered new tserver with Master: af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551)
I20260811 10:09:59.798246 6506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:59895
I20260811 10:09:59.802815 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.795590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.22.177.62:40999)
0811 10:09:59.795708 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.795710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.795763 (+ 53us) server_negotiation.cc:407] Connection header received
0811 10:09:59.795811 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.795812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.795824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.795891 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:09:59.796752 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.797525 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.798498 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.798755 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.798830 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.800048 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:09:59.800049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:09:59.800049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:09:59.800056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:09:59.801716 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.802077 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.802079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.802079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.802146 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:09:59.802334 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:09:59.802334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:09:59.802335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:09:59.802439 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:09:59.802575 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.802628 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.802698 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260811 10:09:59.803812 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:09:59.806851 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.804175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.22.177.62:40999)
0811 10:09:59.804216 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.804218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.804338 (+ 120us) server_negotiation.cc:407] Connection header received
0811 10:09:59.804379 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.804379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.804412 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.804476 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:09:59.804641 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.805102 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.806122 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.806328 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.806386 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.806574 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:09:59.806670 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:09:59.806732 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.806775 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.806797 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:09:59.807137 6506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:09:59.808871 6506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:09:59.811563 6642 tablet_service.cc:1467] Processing CreateTablet for tablet 2574329f3f59408fb000992c07e75319 (DEFAULT_TABLE table=test-table [id=a63f72a41fe64d678ac58efeedbd90f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:59.811796 6642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2574329f3f59408fb000992c07e75319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:59.813359 6596 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Bootstrap starting.
I20260811 10:09:59.813846 6596 tablet_bootstrap.cc:654] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:59.814888 6596 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: No bootstrap required, opened a new log
I20260811 10:09:59.814954 6596 ts_tablet_manager.cc:1397] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:09:59.815119 6596 raft_consensus.cc:348] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.815209 6596 raft_consensus.cc:374] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:59.815274 6596 raft_consensus.cc:729] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a3a59df3548469931bd4765e68aff, State: Initialized, Role: FOLLOWER
I20260811 10:09:59.815325 6596 consensus_queue.cc:260] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.815388 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.810797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:46551 (local address 127.0.0.1:34012)
0811 10:09:59.810851 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.810863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.810911 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.811267 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.811268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.811271 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:59.811326 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.811327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.812722 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.812723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.814094 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.814097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.815083 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.815084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.815148 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.815169 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.815240 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.815311 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":53}
I20260811 10:09:59.815423 6596 ts_tablet_manager.cc:1428] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:09:59.815636 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.810918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.22.177.3:46551)
0811 10:09:59.811088 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.811093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.811103 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:09:59.811147 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.811149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.811161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.811255 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:59.811385 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.812527 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.814196 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.814993 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.815079 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.815145 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.815504 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.815553 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260811 10:09:59.816371 6942 tablet_service.cc:1467] Processing CreateTablet for tablet 2574329f3f59408fb000992c07e75319 (DEFAULT_TABLE table=test-table [id=a63f72a41fe64d678ac58efeedbd90f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:59.816634 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.810845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44293 (local address 127.0.0.1:51132)
0811 10:09:59.811017 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 10:09:59.811028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:09:59.811080 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:09:59.811613 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:09:59.811614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:09:59.811616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:09:59.811664 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.811666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.812783 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.812784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.814094 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:09:59.814097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.816381 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.816382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:09:59.816456 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.816460 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:09:59.816516 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:09:59.816558 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":12,"thread_start_us":114,"threads_started":1}
I20260811 10:09:59.816643 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:09:59.811261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.22.177.2:44293)
0811 10:09:59.811454 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 10:09:59.811458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:09:59.811468 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:09:59.811510 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:09:59.811511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:09:59.811523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:09:59.811614 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:09:59.811969 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.812674 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.815188 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:09:59.816278 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:09:59.816365 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:09:59.816422 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:09:59.816531 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 10:09:59.816568 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":123,"threads_started":1}
I20260811 10:09:59.816781 6942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2574329f3f59408fb000992c07e75319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:59.817425 6796 tablet_service.cc:1467] Processing CreateTablet for tablet 2574329f3f59408fb000992c07e75319 (DEFAULT_TABLE table=test-table [id=a63f72a41fe64d678ac58efeedbd90f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:09:59.817806 6796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2574329f3f59408fb000992c07e75319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:09:59.820612 7027 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Bootstrap starting.
I20260811 10:09:59.821317 7027 tablet_bootstrap.cc:654] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:59.821394 7029 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Bootstrap starting.
I20260811 10:09:59.821936 7027 log.cc:824] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:59.822009 7029 tablet_bootstrap.cc:654] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Neither blocks nor log segments found. Creating new log.
I20260811 10:09:59.822633 7029 log.cc:824] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Log is configured to *not* fsync() on all Append() calls
I20260811 10:09:59.822822 7027 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: No bootstrap required, opened a new log
I20260811 10:09:59.822917 7027 ts_tablet_manager.cc:1397] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:59.823436 7029 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: No bootstrap required, opened a new log
I20260811 10:09:59.823503 7029 ts_tablet_manager.cc:1397] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:59.824561 7027 raft_consensus.cc:348] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.824697 7027 raft_consensus.cc:374] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:59.824720 7027 raft_consensus.cc:729] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4a47638bc24725accd08c6f955a5f0, State: Initialized, Role: FOLLOWER
I20260811 10:09:59.824854 7027 consensus_queue.cc:260] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.824971 7029 raft_consensus.cc:348] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.825125 7029 raft_consensus.cc:374] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:09:59.825151 7027 ts_tablet_manager.cc:1428] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:09:59.825163 7029 raft_consensus.cc:729] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c90c49531d34ae890c64021b91e9ae1, State: Initialized, Role: FOLLOWER
I20260811 10:09:59.825172 7016 heartbeater.cc:503] Master 127.22.177.62:40999 was elected leader, sending a full tablet report...
I20260811 10:09:59.825281 7029 consensus_queue.cc:260] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:09:59.825574 7029 ts_tablet_manager.cc:1428] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:09:59.825666 6871 heartbeater.cc:503] Master 127.22.177.62:40999 was elected leader, sending a full tablet report...
W20260811 10:09:59.892505 6872 tablet.cc:2369] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:10:00.000684 6717 tablet.cc:2369] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:10:00.037317 7018 tablet.cc:2369] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:10:00.095891 6722 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:00.096081 6722 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:00.096575 6722 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
I20260811 10:10:00.100005 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.096946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:42203 (local address 127.22.177.2:44293)
0811 10:10:00.097008 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:00.097011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:00.097035 (+ 24us) server_negotiation.cc:407] Connection header received
0811 10:10:00.097119 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:00.097120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:00.097134 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:00.097216 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:00.097405 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.097987 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.099130 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.099584 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.099682 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:00.099693 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:00.099879 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 10:10:00.099923 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:10:00.100112 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.096898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:36189 (local address 127.22.177.3:46551)
0811 10:10:00.096949 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:00.096952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:00.097007 (+ 55us) server_negotiation.cc:407] Connection header received
0811 10:10:00.097132 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:00.097134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:00.097145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:00.097223 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:00.097371 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.097994 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.099209 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.099670 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.099774 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:00.099784 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:00.099963 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 10:10:00.100007 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:10:00.100015 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.096763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44293 (local address 127.22.177.1:42203)
0811 10:10:00.096918 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.096933 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:00.097025 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:00.097224 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:00.097225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:00.097227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:00.097325 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:00.097326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.098079 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.098080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:00.099020 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:00.099023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.099682 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.099683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:00.099766 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:00.099791 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 10:10:00.099872 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:10:00.099923 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":38,"thread_start_us":81,"threads_started":1}
I20260811 10:10:00.100270 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.096768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:46551 (local address 127.22.177.1:36189)
0811 10:10:00.096931 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.096944 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:00.097040 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:00.097226 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:00.097228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:00.097229 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:00.097296 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:00.097298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.098089 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.098090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:00.099031 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:00.099034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.099803 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.099804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:00.099877 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:00.099881 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:10:00.099956 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:10:00.100004 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260811 10:10:00.100679 6818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2574329f3f59408fb000992c07e75319" candidate_uuid: "014a3a59df3548469931bd4765e68aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c90c49531d34ae890c64021b91e9ae1" is_pre_election: true
I20260811 10:10:00.100824 6964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2574329f3f59408fb000992c07e75319" candidate_uuid: "014a3a59df3548469931bd4765e68aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4a47638bc24725accd08c6f955a5f0" is_pre_election: true
I20260811 10:10:00.100908 6818 raft_consensus.cc:2454] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 014a3a59df3548469931bd4765e68aff in term 0.
I20260811 10:10:00.100965 6964 raft_consensus.cc:2454] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 014a3a59df3548469931bd4765e68aff in term 0.
I20260811 10:10:00.101191 6590 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 014a3a59df3548469931bd4765e68aff, 8c90c49531d34ae890c64021b91e9ae1; no voters:
I20260811 10:10:00.101432 6722 raft_consensus.cc:2781] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:10:00.101503 6722 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:10:00.101524 6722 raft_consensus.cc:3037] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:00.102344 6722 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:00.102504 6722 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 1 election: Requested vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
I20260811 10:10:00.102689 6964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2574329f3f59408fb000992c07e75319" candidate_uuid: "014a3a59df3548469931bd4765e68aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4a47638bc24725accd08c6f955a5f0"
I20260811 10:10:00.102705 6818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2574329f3f59408fb000992c07e75319" candidate_uuid: "014a3a59df3548469931bd4765e68aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c90c49531d34ae890c64021b91e9ae1"
I20260811 10:10:00.102767 6964 raft_consensus.cc:3037] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:00.102793 6818 raft_consensus.cc:3037] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:00.103688 6964 raft_consensus.cc:2454] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 014a3a59df3548469931bd4765e68aff in term 1.
I20260811 10:10:00.103837 6818 raft_consensus.cc:2454] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 014a3a59df3548469931bd4765e68aff in term 1.
I20260811 10:10:00.104038 6590 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 014a3a59df3548469931bd4765e68aff, 8c90c49531d34ae890c64021b91e9ae1; no voters:
I20260811 10:10:00.104210 6722 raft_consensus.cc:2781] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:10:00.104410 6722 raft_consensus.cc:686] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 LEADER]: Becoming Leader. State: Replica: 014a3a59df3548469931bd4765e68aff, State: Running, Role: LEADER
I20260811 10:10:00.104535 6722 consensus_queue.cc:237] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:00.105437 6506 catalog_manager.cc:5495] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff reported cstate change: term changed from 0 to 1, leader changed from <none> to 014a3a59df3548469931bd4765e68aff (127.22.177.1). New cstate: current_term: 1 leader_uuid: "014a3a59df3548469931bd4765e68aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260811 10:10:00.171775 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.169032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.22.177.1:44881)
0811 10:10:00.169214 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:00.169216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:00.169225 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:10:00.169270 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:00.169271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:00.169292 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:00.169369 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:10:00.169505 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.170091 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.170921 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:00.171152 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:00.171214 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:00.171407 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:10:00.171519 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:10:00.171583 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:00.171669 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:10:00.171697 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260811 10:10:00.173727 6964 raft_consensus.cc:1261] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Refusing update from remote peer 014a3a59df3548469931bd4765e68aff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:10:00.173733 6818 raft_consensus.cc:1261] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Refusing update from remote peer 014a3a59df3548469931bd4765e68aff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:10:00.174181 6722 consensus_queue.cc:1035] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:10:00.174335 7037 consensus_queue.cc:1035] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:10:00.178381 7043 mvcc.cc:205] Tried to move back new op lower bound from 7317270528708280320 to 7317270528429101056. Current Snapshot: MvccSnapshot[applied={T|T < 7317270528708280320 or (T in {7317270528708280320})}]
I20260811 10:10:00.178431 7042 mvcc.cc:205] Tried to move back new op lower bound from 7317270528708280320 to 7317270528429101056. Current Snapshot: MvccSnapshot[applied={T|T < 7317270528708280320}]
I20260811 10:10:00.179636 6506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:10:00.179733 6506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:10:00.180580 6506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Sending DeleteTablet for 3 replicas of tablet 2574329f3f59408fb000992c07e75319
I20260811 10:10:00.180943 6942 tablet_service.cc:1514] Processing DeleteTablet for tablet 2574329f3f59408fb000992c07e75319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:00 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:34012
I20260811 10:10:00.181039 6642 tablet_service.cc:1514] Processing DeleteTablet for tablet 2574329f3f59408fb000992c07e75319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:00 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:41068
I20260811 10:10:00.181027 6796 tablet_service.cc:1514] Processing DeleteTablet for tablet 2574329f3f59408fb000992c07e75319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:00 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:51132
I20260811 10:10:00.181268 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 6549
I20260811 10:10:00.181327 7052 tablet_replica.cc:330] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: stopping tablet replica
I20260811 10:10:00.181381 7052 raft_consensus.cc:2229] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:10:00.181483 7052 raft_consensus.cc:2258] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:00.182623 7054 tablet_replica.cc:330] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: stopping tablet replica
I20260811 10:10:00.182734 7054 raft_consensus.cc:2229] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:10:00.182852 7054 raft_consensus.cc:2258] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:00.184351 7054 ts_tablet_manager.cc:1905] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:00.186546 7054 ts_tablet_manager.cc:1918] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:00.186619 7054 log.cc:1196] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/2574329f3f59408fb000992c07e75319
I20260811 10:10:00.186792 7051 tablet_replica.cc:330] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: stopping tablet replica
I20260811 10:10:00.186933 7051 raft_consensus.cc:2229] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:10:00.186965 7054 ts_tablet_manager.cc:1939] T 2574329f3f59408fb000992c07e75319 P 8c90c49531d34ae890c64021b91e9ae1: Deleting consensus metadata
I20260811 10:10:00.187069 7051 raft_consensus.cc:2258] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:00.187529 6485 catalog_manager.cc:4841] TS 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): tablet 2574329f3f59408fb000992c07e75319 (table test-table [id=a63f72a41fe64d678ac58efeedbd90f5]) successfully deleted
I20260811 10:10:00.187583 7051 ts_tablet_manager.cc:1905] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:00.189528 7051 ts_tablet_manager.cc:1918] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:00.189651 7051 log.cc:1196] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/2574329f3f59408fb000992c07e75319
I20260811 10:10:00.189986 7051 ts_tablet_manager.cc:1939] T 2574329f3f59408fb000992c07e75319 P af4a47638bc24725accd08c6f955a5f0: Deleting consensus metadata
I20260811 10:10:00.190860 6488 catalog_manager.cc:4841] TS af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): tablet 2574329f3f59408fb000992c07e75319 (table test-table [id=a63f72a41fe64d678ac58efeedbd90f5]) successfully deleted
W20260811 10:10:00.191983 6485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:44881 (error 108)
I20260811 10:10:00.192490 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 6723
I20260811 10:10:00.192615 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.192442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:44881 (local address 127.0.0.1:41082)
0811 10:10:00.192512 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.192567 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 10:10:00.195508 6485 catalog_manager.cc:4584] TS 014a3a59df3548469931bd4765e68aff (127.22.177.1:44881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2574329f3f59408fb000992c07e75319: Network error: Client connection negotiation failed: client connection to 127.22.177.1:44881: connect: Connection refused (error 111)
I20260811 10:10:00.203459 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 6874
I20260811 10:10:00.210697 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 6465
I20260811 10:10:00.220192 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40999
--webserver_interface=127.22.177.62
--webserver_port=35701
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:00.316229 7057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:00.316514 7057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:00.316560 7057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:00.317873 7057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:10:00.317937 7057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:00.317965 7057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:00.317993 7057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:10:00.318024 7057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:10:00.319636 7057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=35701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:00.319926 7057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:00.320170 7057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:00.326131 7062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.326057 7063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.326154 7065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:00.326414 7057 server_base.cc:1034] running on GCE node
I20260811 10:10:00.326727 7057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:00.327025 7057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:00.328212 7057 hybrid_clock.cc:648] HybridClock initialized: now 1786443000328168 us; error 71 us; skew 500 ppm
Aug 11 10:10:00 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443000, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:00.331151 7057 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:10:00.331696 7057 webserver.cc:460] Webserver started at http://127.22.177.62:35701/ using document root <none> and password file <none>
I20260811 10:10:00.331918 7057 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:00.331964 7057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:00.333400 7057 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.334317 7072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.334692 7057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.334910 7057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "5f26e9e1a9b540a1a51d622806130cd9"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "6cc625e7304510e23851ed3534d30051"
server_key_iv: "3bdef321fb258fff51d05f22ddf288df"
server_key_version: "encryptionkey@0"
I20260811 10:10:00.335440 7057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:00.357059 7057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:00.359429 7057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:00.359613 7057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:00.364199 7057 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40999
I20260811 10:10:00.364269 7137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40999 every 8 connection(s)
I20260811 10:10:00.364684 7057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:10:00.366577 7138 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:10:00.366914 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7057
I20260811 10:10:00.367214 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:44881
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:40999
--webserver_port=34307
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:10:00.367470 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap starting.
I20260811 10:10:00.380213 7138 log.cc:824] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:00.383706 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:10:00.384083 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9: Bootstrap complete.
I20260811 10:10:00.387465 7138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:10:00.387935 7138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f26e9e1a9b540a1a51d622806130cd9, State: Initialized, Role: FOLLOWER
I20260811 10:10:00.388208 7138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:10:00.388309 7138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:10:00.388355 7138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:10:00.388408 7138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 10:10:00.389972 7138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:10:00.390100 7138 leader_election.cc:302] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f26e9e1a9b540a1a51d622806130cd9; no voters:
I20260811 10:10:00.390313 7138 leader_election.cc:288] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 10:10:00.390429 7142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 10:10:00.390604 7142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [term 3 LEADER]: Becoming Leader. State: Replica: 5f26e9e1a9b540a1a51d622806130cd9, State: Running, Role: LEADER
I20260811 10:10:00.390625 7138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:10:00.390776 7142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } }
I20260811 10:10:00.391049 7142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f26e9e1a9b540a1a51d622806130cd9. Latest consensus state: current_term: 3 leader_uuid: "5f26e9e1a9b540a1a51d622806130cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } } }
I20260811 10:10:00.391232 7142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:00.391049 7143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5f26e9e1a9b540a1a51d622806130cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f26e9e1a9b540a1a51d622806130cd9" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40999 } } }
I20260811 10:10:00.391335 7143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f26e9e1a9b540a1a51d622806130cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:00.391616 7149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:10:00.392508 7149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:10:00.392617 7149 catalog_manager.cc:1259] Loaded cluster ID: 72ae889b77a042038ba3a7ffbf58f97a
I20260811 10:10:00.392652 7149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 10:10:00.393121 7149 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f468d7109ad google::LogMessage::Fail() at ??:0
@ 0x7f468d71388a google::LogMessage::SendToLog() at ??:0
@ 0x7f468d7104de google::LogMessage::Flush() at ??:0
@ 0x7f468d711a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f46956350f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f469563e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f468f5de0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f468f5d3954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f46904376db start_thread at ??:0
@ 0x7f468c20e61f clone at ??:0
W20260811 10:10:00.497794 7140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:00.498096 7140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:00.498138 7140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:00.500174 7140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:00.500265 7140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:00.500314 7140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:10:00.502730 7140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:44881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=34307
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:00.503033 7140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:00.503321 7140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:00.509516 7176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.509636 7178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:00.509899 7140 server_base.cc:1034] running on GCE node
W20260811 10:10:00.509546 7175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:00.510223 7140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:00.510555 7140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:00.511749 7140 hybrid_clock.cc:648] HybridClock initialized: now 1786443000511727 us; error 41 us; skew 500 ppm
Aug 11 10:10:00 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443000, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:00.515220 7140 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:10:00.515892 7140 webserver.cc:460] Webserver started at http://127.22.177.1:34307/ using document root <none> and password file <none>
I20260811 10:10:00.516179 7140 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:00.516263 7140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:00.518268 7140 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:10:00.519202 7185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.519449 7140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.519574 7140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "014a3a59df3548469931bd4765e68aff"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "e2f110fc35d69fcedaa3e23aa2c9d02e"
server_key_iv: "57f7bd1f45b233a27da7251c04e46d66"
server_key_version: "encryptionkey@0"
I20260811 10:10:00.520022 7140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:00.543542 7140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:00.546507 7140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:00.546684 7140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:00.547001 7140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:00.547814 7193 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:10:00.549481 7140 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:10:00.549558 7140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:10:00.549621 7140 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:10:00.550621 7140 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:10:00.550676 7140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:10:00.550805 7193 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Bootstrap starting.
I20260811 10:10:00.558719 7140 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:44881
I20260811 10:10:00.559010 7311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:44881 every 8 connection(s)
I20260811 10:10:00.559247 7140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:10:00.562449 7193 log.cc:824] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:00.564958 7193 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:10:00.565284 7193 tablet_bootstrap.cc:492] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Bootstrap complete.
I20260811 10:10:00.566175 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7140
I20260811 10:10:00.566499 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:44293
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:40999
--webserver_port=39183
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:10:00.566622 7193 ts_tablet_manager.cc:1397] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260811 10:10:00.568614 7193 raft_consensus.cc:348] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:00.568913 7193 raft_consensus.cc:729] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a3a59df3548469931bd4765e68aff, State: Initialized, Role: FOLLOWER
I20260811 10:10:00.569099 7193 consensus_queue.cc:260] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:00.569438 7193 ts_tablet_manager.cc:1428] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260811 10:10:00.633303 7314 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:10:00.560095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:55695)
0811 10:10:00.560350 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.560414 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:00.561057 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:00.632999 (+ 71942us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":126,"threads_started":1}
W20260811 10:10:00.633559 7312 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:40999 (0 consecutive failures): Network error: Failed to ping master at 127.22.177.62:40999: Client connection negotiation failed: client connection to 127.22.177.62:40999: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:10:00.633965 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.633833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:49269)
0811 10:10:00.633894 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.633941 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:10:00.634485 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.634303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:58211)
0811 10:10:00.634423 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.634462 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 10:10:00.634688 7312 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:10:00.665206 7316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:00.665477 7316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:00.665522 7316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:00.666837 7316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:00.666923 7316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:00.666973 7316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:10:00.668614 7316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:44293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=39183
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:00.668920 7316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:00.669178 7316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:00.675096 7325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.675097 7323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.675096 7322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:00.675295 7316 server_base.cc:1034] running on GCE node
I20260811 10:10:00.675628 7316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:00.675940 7316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:00.677170 7316 hybrid_clock.cc:648] HybridClock initialized: now 1786443000677150 us; error 54 us; skew 500 ppm
Aug 11 10:10:00 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443000, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:00.680477 7316 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:10:00.681022 7316 webserver.cc:460] Webserver started at http://127.22.177.2:39183/ using document root <none> and password file <none>
I20260811 10:10:00.681270 7316 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:00.681342 7316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:00.682960 7316 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.683771 7332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.684129 7316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.684249 7316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "8c90c49531d34ae890c64021b91e9ae1"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "02c1949c8ac881dd0d338042d0984768"
server_key_iv: "887e4cd781856f9d8b1c6dea2289fc6c"
server_key_version: "encryptionkey@0"
I20260811 10:10:00.684666 7316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:00.695106 7316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:00.697542 7316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:00.697781 7316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:00.698067 7316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:00.698421 7316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:00.698477 7316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.698527 7316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:00.698557 7316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.705240 7316 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:44293
I20260811 10:10:00.705329 7460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:44293 every 8 connection(s)
I20260811 10:10:00.706063 7316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:10:00.707551 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.706943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:56997)
0811 10:10:00.707099 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.707184 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
W20260811 10:10:00.707834 7461 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:40999 (0 consecutive failures): Network error: Failed to ping master at 127.22.177.62:40999: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
I20260811 10:10:00.708112 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.708034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:56691)
0811 10:10:00.708066 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.708093 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260811 10:10:00.708489 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.708342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:49837)
0811 10:10:00.708423 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.708454 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 10:10:00.708633 7461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:10:00.714421 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7316
I20260811 10:10:00.714763 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:46551
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:40999
--webserver_port=43461
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:00.806098 7465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:00.806361 7465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:00.806391 7465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:00.807701 7465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:00.807756 7465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:00.807787 7465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:10:00.809407 7465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:46551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=43461
--tserver_master_addrs=127.22.177.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:00.809708 7465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:00.809971 7465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:00.815462 7470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.815470 7471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:00.815472 7473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:00.815830 7465 server_base.cc:1034] running on GCE node
I20260811 10:10:00.816056 7465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:00.816331 7465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:00.817502 7465 hybrid_clock.cc:648] HybridClock initialized: now 1786443000817494 us; error 46 us; skew 500 ppm
Aug 11 10:10:00 dist-test-slave-g350 krb5kdc[5850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443000, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:00.820353 7465 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:10:00.820937 7465 webserver.cc:460] Webserver started at http://127.22.177.3:43461/ using document root <none> and password file <none>
I20260811 10:10:00.821259 7465 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:00.821347 7465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:00.823012 7465 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.823772 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.824086 7465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:00.824215 7465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "af4a47638bc24725accd08c6f955a5f0"
format_stamp: "Formatted at 2026-08-11 10:09:58 on dist-test-slave-g350"
server_key: "49e022da9607929b254e985baa4f3d8c"
server_key_iv: "6bbe9a5a6a82271fc581c108a3bbe973"
server_key_version: "encryptionkey@0"
I20260811 10:10:00.824612 7465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:00.839900 7465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:00.842796 7465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:00.842988 7465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:00.843288 7465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:00.843644 7465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:00.843698 7465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.843741 7465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:00.843772 7465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:00.851899 7465 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:46551
I20260811 10:10:00.851980 7609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:46551 every 8 connection(s)
I20260811 10:10:00.852383 7465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:10:00.853708 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.853164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:54829)
0811 10:10:00.853405 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.853510 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
W20260811 10:10:00.854180 7610 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:40999 (0 consecutive failures): Network error: Failed to ping master at 127.22.177.62:40999: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
I20260811 10:10:00.854674 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.854525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:56847)
0811 10:10:00.854600 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.854641 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 10:10:00.855450 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:00.855283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:40549)
0811 10:10:00.855396 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:00.855426 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 10:10:00.855648 7610 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:10:00.862104 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7465
I20260811 10:10:01.635710 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.635183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:36733)
0811 10:10:01.635568 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:01.635617 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":287,"threads_started":1}
I20260811 10:10:01.709326 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.709027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:33853)
0811 10:10:01.709173 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:01.709267 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260811 10:10:01.770792 7617 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:01.770933 7617 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:01.771450 7617 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
Aug 11 10:10:01 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443000, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.3@KRBTEST.COM
Aug 11 10:10:01 dist-test-slave-g350 krb5kdc[5850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443000, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.2@KRBTEST.COM
I20260811 10:10:01.783638 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.771796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:57875 (local address 127.22.177.3:46551)
0811 10:10:01.771934 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:01.771946 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:01.771971 (+ 25us) server_negotiation.cc:407] Connection header received
0811 10:10:01.772548 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:01.772552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:01.772626 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:01.772800 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:01.773849 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.774900 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.775984 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.776422 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.776583 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:01.779261 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:01.779283 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:01.779284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:01.779299 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:01.781766 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782278 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:01.782282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:01.782283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:01.782390 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782664 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:01.782665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:01.782666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:01.782948 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:01.783049 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:01.783359 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 10:10:01.783505 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260811 10:10:01.783627 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.771646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:46551 (local address 127.22.177.1:57875)
0811 10:10:01.771730 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:01.771744 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:01.771811 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:01.772891 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:01.772894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:01.773426 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:01.773674 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:01.773682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.775038 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.775039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:01.775886 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:01.775889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.776553 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.776555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:01.776659 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:01.777307 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:01.777319 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:01.779110 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:01.781886 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:01.781897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:01.781900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:01.782178 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:01.782467 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:01.782469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:01.782591 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:01.783026 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:01.783034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:01.783169 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 10:10:01.783297 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 10:10:01.783516 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260811 10:10:01.783663 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.771823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:37399 (local address 127.22.177.2:44293)
0811 10:10:01.771989 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:01.771997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:01.772026 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:10:01.772571 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:01.772574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:01.772626 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:01.772791 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:01.773860 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.775167 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.775991 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.776419 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.776546 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:01.779615 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:01.779622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:01.779622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:01.779638 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:01.782089 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782502 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:01.782505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:01.782506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:01.782608 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782825 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:01.782827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:01.782827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:01.783067 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:01.783162 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:01.783363 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 10:10:01.783504 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260811 10:10:01.784101 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.771717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44293 (local address 127.22.177.1:37399)
0811 10:10:01.771896 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:01.771908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:01.771966 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:01.772891 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:01.772894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:01.773374 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:01.773674 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:01.773682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.775261 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.775262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:01.775893 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:01.775896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:01.776523 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:01.776524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:01.776656 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:01.777342 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:01.777347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:01.779488 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:01.782205 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:01.782206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:01.782408 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:01.782692 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:01.782693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:01.782693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:01.782758 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:01.783145 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:01.783146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:01.783195 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:10:01.783300 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 10:10:01.783611 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":193,"thread_start_us":112,"threads_started":1}
W20260811 10:10:01.784693 7188 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
W20260811 10:10:01.785023 7187 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
I20260811 10:10:01.785101 7187 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 014a3a59df3548469931bd4765e68aff; no voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0
I20260811 10:10:01.785241 7617 raft_consensus.cc:2726] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:10:01.856454 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:01.856139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:50143)
0811 10:10:01.856343 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:01.856401 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":103,"threads_started":1}
I20260811 10:10:02.636575 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:02.636329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:34859)
0811 10:10:02.636481 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:02.636524 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260811 10:10:02.710300 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:02.709912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:53099)
0811 10:10:02.710070 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:02.710158 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260811 10:10:02.857651 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:02.857217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:50831)
0811 10:10:02.857472 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:02.857521 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":160,"threads_started":1}
I20260811 10:10:03.637630 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:03.637311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:40949)
0811 10:10:03.637504 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:03.637555 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
I20260811 10:10:03.711174 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:03.710908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:40489)
0811 10:10:03.711079 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:03.711125 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260811 10:10:03.858613 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:03.858289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:49865)
0811 10:10:03.858492 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:03.858545 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":109,"threads_started":1}
I20260811 10:10:03.901304 7632 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:03.901456 7632 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:03.901706 7632 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
W20260811 10:10:03.902148 7187 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
W20260811 10:10:03.902243 7188 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
I20260811 10:10:03.902390 7188 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 014a3a59df3548469931bd4765e68aff; no voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0
I20260811 10:10:03.902500 7632 raft_consensus.cc:2726] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:10:04.638463 7633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:04.638200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:46557)
0811 10:10:04.638369 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:04.638414 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":121,"threads_started":1}
I20260811 10:10:04.712344 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:04.711997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:46711)
0811 10:10:04.712224 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:04.712277 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":77,"threads_started":1}
I20260811 10:10:04.859527 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:04.859237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:35553)
0811 10:10:04.859413 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:04.859474 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260811 10:10:05.639464 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:05.639171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:37059)
0811 10:10:05.639362 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:05.639412 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":132,"threads_started":1}
I20260811 10:10:05.713162 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:05.712904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:53335)
0811 10:10:05.713068 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:05.713112 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260811 10:10:05.860507 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:05.860195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:57937)
0811 10:10:05.860382 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:05.860429 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260811 10:10:06.640523 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:06.640212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:54885)
0811 10:10:06.640417 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:06.640476 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260811 10:10:06.713999 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:06.713756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:50569)
0811 10:10:06.713909 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:06.713950 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260811 10:10:06.861495 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:06.861182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:51145)
0811 10:10:06.861397 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:06.861446 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260811 10:10:07.200577 7644 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:07.200733 7644 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:07.200984 7644 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
W20260811 10:10:07.201504 7188 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
W20260811 10:10:07.201567 7187 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
I20260811 10:10:07.201598 7187 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 014a3a59df3548469931bd4765e68aff; no voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0
I20260811 10:10:07.201701 7644 raft_consensus.cc:2726] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:10:07.641503 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:07.641208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:41311)
0811 10:10:07.641402 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:07.641453 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
I20260811 10:10:07.715056 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:07.714766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:33727)
0811 10:10:07.714952 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:07.715006 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
I20260811 10:10:07.862856 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:07.862381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:43759)
0811 10:10:07.862707 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:07.862784 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":143,"threads_started":1}
I20260811 10:10:08.642558 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:08.642209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:50223)
0811 10:10:08.642436 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:08.642493 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
I20260811 10:10:08.716131 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:08.715717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:52059)
0811 10:10:08.716027 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:08.716081 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":193,"threads_started":1}
I20260811 10:10:08.836889 7650 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:08.837016 7650 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:08.837244 7650 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
W20260811 10:10:08.837693 7188 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
W20260811 10:10:08.837785 7187 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
I20260811 10:10:08.837867 7187 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 014a3a59df3548469931bd4765e68aff; no voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0
I20260811 10:10:08.837965 7650 raft_consensus.cc:2726] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:10:08.863739 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:08.863485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:51971)
0811 10:10:08.863647 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:08.863689 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
I20260811 10:10:09.643496 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:09.643219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:35325)
0811 10:10:09.643380 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:09.643441 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260811 10:10:09.717147 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:09.716823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:35255)
0811 10:10:09.717022 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:09.717077 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260811 10:10:09.864622 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:09.864347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:53007)
0811 10:10:09.864524 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:09.864571 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
I20260811 10:10:10.644411 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:10.644158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:43739)
0811 10:10:10.644318 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:10.644363 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260811 10:10:10.718017 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:10.717757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:33395)
0811 10:10:10.717930 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:10.717972 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260811 10:10:10.865731 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:10.865355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:56047)
0811 10:10:10.865542 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:10.865665 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260811 10:10:11.009966 7660 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:11.010070 7660 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:11.010311 7660 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
W20260811 10:10:11.010768 7187 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
W20260811 10:10:11.010855 7188 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
I20260811 10:10:11.010924 7188 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 014a3a59df3548469931bd4765e68aff; no voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0
I20260811 10:10:11.011034 7660 raft_consensus.cc:2726] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:10:11.645409 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:11.645109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:45713)
0811 10:10:11.645289 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:11.645345 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260811 10:10:11.718936 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:11.718690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:57235)
0811 10:10:11.718836 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:11.718884 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260811 10:10:11.866696 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:11.866384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:52769)
0811 10:10:11.866551 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:11.866599 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260811 10:10:12.646329 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:12.646086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:38717)
0811 10:10:12.646246 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:12.646281 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260811 10:10:12.719815 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:12.719544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:38509)
0811 10:10:12.719696 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:12.719742 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260811 10:10:12.867849 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:12.867465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:51953)
0811 10:10:12.867696 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:12.867758 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
I20260811 10:10:13.647369 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:13.647021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:36743)
0811 10:10:13.647270 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:13.647319 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":88,"threads_started":1}
I20260811 10:10:13.720831 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:13.720541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:57999)
0811 10:10:13.720723 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:13.720770 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260811 10:10:13.868784 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:13.868520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:52843)
0811 10:10:13.868682 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:13.868725 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260811 10:10:14.648330 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:14.647964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:37349)
0811 10:10:14.648204 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:14.648257 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":91,"threads_started":1}
I20260811 10:10:14.721722 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:14.721327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:40721)
0811 10:10:14.721506 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:14.721554 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260811 10:10:14.869709 7674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:14.869392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:41327)
0811 10:10:14.869568 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:14.869653 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260811 10:10:15.649443 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:15.649145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.1:59145)
0811 10:10:15.649342 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:15.649389 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":88,"threads_started":1}
I20260811 10:10:15.722697 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:15.722422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.2:39191)
0811 10:10:15.722586 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:15.722642 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260811 10:10:15.730988 7677 raft_consensus.cc:482] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:15.731096 7677 raft_consensus.cc:504] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4a47638bc24725accd08c6f955a5f0" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 46551 } } peers { permanent_uuid: "8c90c49531d34ae890c64021b91e9ae1" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44293 } } peers { permanent_uuid: "014a3a59df3548469931bd4765e68aff" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 44881 } }
I20260811 10:10:15.731251 7677 leader_election.cc:288] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551), 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293)
W20260811 10:10:15.731768 7187 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c90c49531d34ae890c64021b91e9ae1 (127.22.177.2:44293): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
W20260811 10:10:15.731873 7188 leader_election.cc:341] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af4a47638bc24725accd08c6f955a5f0 (127.22.177.3:46551): Not found: Tablet not found: 2574329f3f59408fb000992c07e75319
I20260811 10:10:15.731954 7188 leader_election.cc:302] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 014a3a59df3548469931bd4765e68aff; no voters: 8c90c49531d34ae890c64021b91e9ae1, af4a47638bc24725accd08c6f955a5f0
I20260811 10:10:15.732069 7677 raft_consensus.cc:2726] T 2574329f3f59408fb000992c07e75319 P 014a3a59df3548469931bd4765e68aff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:10:15.867360 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 7140
I20260811 10:10:15.871966 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:15.871171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:40999 (local address 127.22.177.3:57113)
0811 10:10:15.871865 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:15.871912 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":313,"threads_started":1}
I20260811 10:10:15.876914 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 7316
I20260811 10:10:15.883944 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 7465
2026-08-11T10:10:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18977 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:10:15 dist-test-slave-g350 krb5kdc[7682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:10:15 dist-test-slave-g350 krb5kdc[7682](info): set up 2 sockets
Aug 11 10:10:15 dist-test-slave-g350 krb5kdc[7682](info): commencing operation
krb5kdc: starting...
W20260811 10:10:17.029048 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:10:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:10:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:10:17.177297 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36473
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:17.265928 7696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:17.266209 7696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:17.266251 7696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:17.267536 7696 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 10:10:17.267622 7696 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 10:10:17.267643 7696 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 10:10:17.267664 7696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:10:17.267693 7696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:17.267727 7696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:17.267762 7696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:10:17.267791 7696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:10:17.269564 7696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:17.269965 7696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:17.270236 7696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:17.276829 7702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.276867 7706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.276867 7703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:17.276898 7696 server_base.cc:1034] running on GCE node
I20260811 10:10:17.277504 7696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:17.277904 7696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:17.279085 7696 hybrid_clock.cc:648] HybridClock initialized: now 1786443017279047 us; error 57 us; skew 500 ppm
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:17.282159 7696 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:10:17.282720 7696 webserver.cc:460] Webserver started at http://127.22.177.62:45219/ using document root <none> and password file <none>
I20260811 10:10:17.283028 7696 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:17.283105 7696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:17.283282 7696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:10:17.284605 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "e801e94c32214343bc4c7d80cde52a67"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b6208ef8eeae709b4c608ded6f38b0f5"
server_key_iv: "4927699a2ecf8f612e75880f56992277"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.285156 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "e801e94c32214343bc4c7d80cde52a67"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b6208ef8eeae709b4c608ded6f38b0f5"
server_key_iv: "4927699a2ecf8f612e75880f56992277"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.286755 7696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 10:10:17.287709 7714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.287961 7696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:17.288086 7696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e801e94c32214343bc4c7d80cde52a67"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b6208ef8eeae709b4c608ded6f38b0f5"
server_key_iv: "4927699a2ecf8f612e75880f56992277"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.288254 7696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:17.312408 7696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:17.315224 7696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:17.315419 7696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:17.320152 7696 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:36473
I20260811 10:10:17.320459 7771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:36473 every 8 connection(s)
I20260811 10:10:17.320631 7696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:10:17.321583 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:17.324355 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7696
I20260811 10:10:17.324478 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:10:17.324622 7772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap starting.
I20260811 10:10:17.324688 23236 external_mini_cluster.cc:1428] Setting key 9c0aa4d2c4845ab1664aa7c745129adf
I20260811 10:10:17.325428 7772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:17.325834 7772 log.cc:824] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:17.326941 7772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: No bootstrap required, opened a new log
I20260811 10:10:17.329398 7772 raft_consensus.cc:348] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:17.329661 7772 raft_consensus.cc:374] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:17.329713 7772 raft_consensus.cc:729] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e801e94c32214343bc4c7d80cde52a67, State: Initialized, Role: FOLLOWER
I20260811 10:10:17.329872 7772 consensus_queue.cc:260] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:17.329941 7772 raft_consensus.cc:388] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:10:17.329995 7772 raft_consensus.cc:482] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:10:17.330044 7772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:17.330933 7772 raft_consensus.cc:504] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:17.331082 7772 leader_election.cc:302] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e801e94c32214343bc4c7d80cde52a67; no voters:
I20260811 10:10:17.331269 7772 leader_election.cc:288] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:10:17.331493 7779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:10:17.331614 7772 sys_catalog.cc:564] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:10:17.332108 7779 raft_consensus.cc:686] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 LEADER]: Becoming Leader. State: Replica: e801e94c32214343bc4c7d80cde52a67, State: Running, Role: LEADER
I20260811 10:10:17.332311 7779 consensus_queue.cc:237] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:17.332713 7778 sys_catalog.cc:455] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e801e94c32214343bc4c7d80cde52a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } } }
I20260811 10:10:17.332799 7778 sys_catalog.cc:458] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: This master's current role is: LEADER
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:10:17.332959 7780 sys_catalog.cc:455] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e801e94c32214343bc4c7d80cde52a67. Latest consensus state: current_term: 1 leader_uuid: "e801e94c32214343bc4c7d80cde52a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } } }
I20260811 10:10:17.333039 7780 sys_catalog.cc:458] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:17.333197 7788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:10:17.334079 7788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:10:17.335903 7788 catalog_manager.cc:1347] Generated new cluster ID: 9e693eb8f00b4060a76f2cc2fb01e65e
I20260811 10:10:17.335981 7788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:10:17.337814 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.325291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.22.177.62:36473)
0811 10:10:17.325480 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.325490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.325528 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:10:17.326164 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.326174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.326257 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.326430 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:17.327311 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.328247 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.329489 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.330244 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.330355 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.333442 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:17.333447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:17.333447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:17.333460 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:17.335566 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.336048 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.336052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.336052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.336158 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.336397 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.336398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.336399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.336718 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:17.336867 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.337384 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.337537 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260811 10:10:17.346801 7788 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:10:17.347369 7788 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:10:17.349999 7788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Generated new TSK 0
I20260811 10:10:17.350351 7788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:10:17.390020 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:36473
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:17.480406 7803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:17.480693 7803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:17.480736 7803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:17.482069 7803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:17.482151 7803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:17.482198 7803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:10:17.483865 7803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:17.484187 7803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:17.484465 7803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:17.490617 7811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.490617 7808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.490612 7813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:17.490876 7803 server_base.cc:1034] running on GCE node
I20260811 10:10:17.491133 7803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:17.491499 7803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:17.492708 7803 hybrid_clock.cc:648] HybridClock initialized: now 1786443017492673 us; error 49 us; skew 500 ppm
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:17.495895 7803 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:10:17.496404 7803 webserver.cc:460] Webserver started at http://127.22.177.1:35227/ using document root <none> and password file <none>
I20260811 10:10:17.496673 7803 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:17.496747 7803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:17.496943 7803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:10:17.498215 7803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "2389cdd77e2344a189a8b6e456aeee3e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b264482a2a9e261163294aefee27657c"
server_key_iv: "e00c66a5d1a30941b39a18af98e406ea"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.498634 7803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "2389cdd77e2344a189a8b6e456aeee3e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b264482a2a9e261163294aefee27657c"
server_key_iv: "e00c66a5d1a30941b39a18af98e406ea"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.500185 7803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:10:17.501168 7821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.501462 7803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:17.501639 7803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "2389cdd77e2344a189a8b6e456aeee3e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b264482a2a9e261163294aefee27657c"
server_key_iv: "e00c66a5d1a30941b39a18af98e406ea"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.501744 7803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:17.525709 7803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:17.528370 7803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:17.528556 7803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:17.528831 7803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:17.529183 7803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:17.529240 7803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.529289 7803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:17.529316 7803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.536314 7803 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:36919
I20260811 10:10:17.536358 7952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:36919 every 8 connection(s)
I20260811 10:10:17.536751 7803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:10:17.537421 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7803
I20260811 10:10:17.537562 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:10:17.537847 23236 external_mini_cluster.cc:1428] Setting key 984e620000b40c3b490360c5c40d4f56
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
I20260811 10:10:17.548929 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.537730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:59493 (local address 127.22.177.62:36473)
0811 10:10:17.537800 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.537802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.538103 (+ 301us) server_negotiation.cc:407] Connection header received
0811 10:10:17.538721 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.538722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.538736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.538781 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:17.540205 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.540863 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.542040 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.542269 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.542362 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.545181 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:17.545183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:17.545183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:17.545191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:17.547063 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.547672 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.547673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.547674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.547743 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.548069 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.548070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.548070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.548280 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:17.548384 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.548704 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.548813 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:10:17.549193 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.537646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:59493)
0811 10:10:17.537944 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:17.537993 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:17.538631 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:17.538871 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:17.538875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:17.539785 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:17.540079 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.540085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.540982 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.540984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.541938 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.541941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.542379 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.542380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.542512 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.543172 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:17.543183 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:17.545030 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:17.547176 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:17.547188 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:17.547191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:17.547570 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:17.547827 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:17.547828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:17.547829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:17.547991 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:17.548385 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:17.548394 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:17.548559 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 10:10:17.548685 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 10:10:17.548924 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":116,"threads_started":1}
I20260811 10:10:17.550069 7953 heartbeater.cc:347] Connected to a master server at 127.22.177.62:36473
I20260811 10:10:17.550179 7953 heartbeater.cc:464] Registering TS with master...
I20260811 10:10:17.550429 7953 heartbeater.cc:511] Master 127.22.177.62:36473 requested a full tablet report, sending...
I20260811 10:10:17.551534 7734 ts_manager.cc:194] Registered new tserver with Master: 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
I20260811 10:10:17.552716 7734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:59493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:10:17.574039 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:36473
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:17.663230 7958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:17.663532 7958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:17.663576 7958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:17.664924 7958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:17.665014 7958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:17.665077 7958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:10:17.666790 7958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:17.667114 7958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:17.667398 7958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:17.673475 7967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.673467 7964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.673467 7965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:17.673514 7958 server_base.cc:1034] running on GCE node
I20260811 10:10:17.674049 7958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:17.674324 7958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:17.675509 7958 hybrid_clock.cc:648] HybridClock initialized: now 1786443017675493 us; error 41 us; skew 500 ppm
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:17.683807 7958 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:10:17.684479 7958 webserver.cc:460] Webserver started at http://127.22.177.2:35585/ using document root <none> and password file <none>
I20260811 10:10:17.684741 7958 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:17.684816 7958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:17.684984 7958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:10:17.686260 7958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "81663a104e7248f18ef16bcb25ebf08e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "7e7aa53d365e276690aabaf689ac1546"
server_key_iv: "3bfb97b9ea2fc8f9ae8e4ad502f884df"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.686693 7958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "81663a104e7248f18ef16bcb25ebf08e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "7e7aa53d365e276690aabaf689ac1546"
server_key_iv: "3bfb97b9ea2fc8f9ae8e4ad502f884df"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.688509 7958 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 10:10:17.689471 7974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.689728 7958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:17.689847 7958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "81663a104e7248f18ef16bcb25ebf08e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "7e7aa53d365e276690aabaf689ac1546"
server_key_iv: "3bfb97b9ea2fc8f9ae8e4ad502f884df"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.689950 7958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:17.707046 7958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:17.709744 7958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:17.709931 7958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:17.710227 7958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:17.710579 7958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:17.710636 7958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.710707 7958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:17.710737 7958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.717366 7958 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35823
I20260811 10:10:17.717425 8099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35823 every 8 connection(s)
I20260811 10:10:17.717856 7958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:10:17.721000 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 7958
I20260811 10:10:17.721161 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:10:17.721388 23236 external_mini_cluster.cc:1428] Setting key 54508f171c740d4cba8090dca3863f6c
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:10:17.729108 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.718785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:58829 (local address 127.22.177.62:36473)
0811 10:10:17.718860 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.718862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.719100 (+ 238us) server_negotiation.cc:407] Connection header received
0811 10:10:17.719710 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.719711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.719723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.719813 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:17.720864 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.721552 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.722801 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.723028 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.723129 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.725947 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:17.725949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:17.725949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:17.725960 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:17.727516 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.728018 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.728019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.728020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.728084 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.728361 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.728363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.728363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.728495 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:17.728571 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.728907 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.729007 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:10:17.729339 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.718748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:58829)
0811 10:10:17.718924 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:17.718975 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:17.719574 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:17.719891 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:17.719894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:17.720525 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:17.720733 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.720738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.721725 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.721726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.722699 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.722702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.723146 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.723147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.723305 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.723922 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:17.723931 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:17.725822 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:17.727641 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:17.727653 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:17.727656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:17.727924 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:17.728156 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:17.728157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:17.728158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:17.728287 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:17.728573 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:17.728582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:17.728742 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 10:10:17.728877 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 10:10:17.729068 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260811 10:10:17.729914 8100 heartbeater.cc:347] Connected to a master server at 127.22.177.62:36473
I20260811 10:10:17.730012 8100 heartbeater.cc:464] Registering TS with master...
I20260811 10:10:17.730247 8100 heartbeater.cc:511] Master 127.22.177.62:36473 requested a full tablet report, sending...
I20260811 10:10:17.730744 7734 ts_manager.cc:194] Registered new tserver with Master: 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823)
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:10:17.731209 7734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:58829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:10:17.759660 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:36473
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:17.854833 8105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:17.855093 8105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:17.855122 8105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:17.856540 8105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:17.856606 8105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:17.856638 8105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:10:17.858386 8105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:17.858621 8105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:17.858837 8105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:17.865139 8112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.865356 8117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:17.865139 8114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:17.865420 8105 server_base.cc:1034] running on GCE node
I20260811 10:10:17.865826 8105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:17.866146 8105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:17.867416 8105 hybrid_clock.cc:648] HybridClock initialized: now 1786443017867391 us; error 48 us; skew 500 ppm
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:17.870675 8105 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:10:17.871392 8105 webserver.cc:460] Webserver started at http://127.22.177.3:38361/ using document root <none> and password file <none>
I20260811 10:10:17.871701 8105 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:17.871798 8105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:17.871974 8105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:10:17.873209 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "1bd45df631ba4211b9d9d902c53beefd"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "dbf46e03a9d30820bd92e8581192a414"
server_key_iv: "88029766e602d82568d361c693e1af19"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.873687 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "1bd45df631ba4211b9d9d902c53beefd"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "dbf46e03a9d30820bd92e8581192a414"
server_key_iv: "88029766e602d82568d361c693e1af19"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.875360 8105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:10:17.876371 8125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.876626 8105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:10:17.876751 8105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "1bd45df631ba4211b9d9d902c53beefd"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "dbf46e03a9d30820bd92e8581192a414"
server_key_iv: "88029766e602d82568d361c693e1af19"
server_key_version: "encryptionkey@0"
I20260811 10:10:17.876860 8105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:17.896507 8105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:17.899223 8105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:17.899418 8105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:17.899693 8105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:17.900058 8105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:17.900116 8105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.900153 8105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:17.900202 8105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:17.906865 8105 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:45295
I20260811 10:10:17.906921 8250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:45295 every 8 connection(s)
I20260811 10:10:17.907275 8105 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:10:17 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:10:17.916944 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8105
I20260811 10:10:17.917098 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:10:17.917310 23236 external_mini_cluster.cc:1428] Setting key f1de442983f9220a97b8c2723bb88e3e
I20260811 10:10:17.917584 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.908085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:34545 (local address 127.22.177.62:36473)
0811 10:10:17.908165 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.908166 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.908475 (+ 309us) server_negotiation.cc:407] Connection header received
0811 10:10:17.909034 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.909036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.909050 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.909104 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:17.910208 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.910724 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.911769 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.911960 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.912054 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.914498 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:17.914500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:17.914500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:17.914509 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:17.916052 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.916496 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.916497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.916498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.916559 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.916846 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.916847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.916848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.916975 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:17.917054 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.917391 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.917482 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:10:17.917846 8254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.908064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:34545)
0811 10:10:17.908325 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:17.908370 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:17.908907 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:17.909188 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:17.909191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:17.909891 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:17.910096 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.910100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.910845 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.910846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.911631 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.911634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.912080 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.912081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.912205 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.912776 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:17.912785 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:17.914393 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:17.916148 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:17.916157 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:17.916160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:17.916411 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:17.916629 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:17.916630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:17.916631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:17.916776 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:17.917045 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:17.917052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:17.917234 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 10:10:17.917370 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 10:10:17.917561 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":148,"threads_started":1}
I20260811 10:10:17.918396 8251 heartbeater.cc:347] Connected to a master server at 127.22.177.62:36473
I20260811 10:10:17.918525 8251 heartbeater.cc:464] Registering TS with master...
I20260811 10:10:17.918707 8251 heartbeater.cc:511] Master 127.22.177.62:36473 requested a full tablet report, sending...
I20260811 10:10:17.919065 7734 ts_manager.cc:194] Registered new tserver with Master: 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295)
I20260811 10:10:17.919466 7734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:34545
I20260811 10:10:17.929898 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:10:17.938583 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.931531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.22.177.62:36473)
0811 10:10:17.931582 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.931584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.931591 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:10:17.931613 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.931614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.931623 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.931677 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:17.932458 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.932978 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.933917 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.934122 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.934191 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.935742 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:17.935743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:17.935744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:17.935752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:17.937244 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.937681 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.937682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.937683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.937773 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:17.938026 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:17.938027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:17.938028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:17.938198 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:17.938305 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.938412 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.938482 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:10:17.941036 7734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:10:17.943758 7734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:10:17.952579 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.948160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.22.177.1:36919)
0811 10:10:17.948364 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.948368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.948381 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:10:17.948421 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.948423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.948435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.948976 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:17.949119 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.950054 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.951563 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.952089 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.952190 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.952269 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.952453 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.952497 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260811 10:10:17.952572 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.947971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45295 (local address 127.0.0.1:56244)
0811 10:10:17.948079 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:17.948087 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:17.948127 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:17.948905 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:17.948906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:17.948909 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:17.948950 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.948952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.950185 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.950187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.951227 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.951230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.952299 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.952300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.952371 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.952375 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:10:17.952422 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 10:10:17.952456 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":45,"thread_start_us":71,"threads_started":1}
I20260811 10:10:17.952776 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.947913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.22.177.2:35823)
0811 10:10:17.948129 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.948135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.948144 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:10:17.948187 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.948190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.948205 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.948280 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:17.948479 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.949726 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.951397 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.952124 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.952518 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.952593 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.952622 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.952684 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":123,"threads_started":1}
I20260811 10:10:17.952998 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.948182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.22.177.3:45295)
0811 10:10:17.948658 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:17.948662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:17.948670 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:10:17.948745 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:17.948747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:17.948762 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:17.948836 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:17.949036 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.950073 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.951320 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.952124 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.952207 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.952268 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:17.952825 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 10:10:17.952875 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":85,"threads_started":1}
I20260811 10:10:17.953585 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.947672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35823 (local address 127.0.0.1:44766)
0811 10:10:17.947810 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:17.947856 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:17.947937 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:17.948306 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:17.948310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:17.948312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:17.948391 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.948393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.949843 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.949845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.951318 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.951321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.952239 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.952240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.952324 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.952328 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:10:17.952372 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 10:10:17.952449 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":76,"thread_start_us":91,"threads_started":1}
I20260811 10:10:17.952579 8267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:17.947702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36919 (local address 127.0.0.1:35742)
0811 10:10:17.947887 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:17.947897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:17.947940 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:17.948973 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:17.948974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:17.948977 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:17.949050 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.949052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.950198 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.950199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.951482 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:17.951484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:17.952192 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:17.952193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:17.952298 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:17.952314 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 10:10:17.952430 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 10:10:17.952479 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":147,"threads_started":1}
I20260811 10:10:17.954118 8182 tablet_service.cc:1467] Processing CreateTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb (DEFAULT_TABLE table=test-table [id=30bc459f66624f7883b903d743a130b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:10:17.954412 8029 tablet_service.cc:1467] Processing CreateTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb (DEFAULT_TABLE table=test-table [id=30bc459f66624f7883b903d743a130b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:10:17.954455 7875 tablet_service.cc:1467] Processing CreateTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb (DEFAULT_TABLE table=test-table [id=30bc459f66624f7883b903d743a130b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:10:17.954694 8029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba97d3b33be44288a0afbf6fc39dcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:17.954686 8182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba97d3b33be44288a0afbf6fc39dcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:17.954767 7875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba97d3b33be44288a0afbf6fc39dcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:17.957482 8274 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Bootstrap starting.
I20260811 10:10:17.958340 8276 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap starting.
I20260811 10:10:17.958560 8274 tablet_bootstrap.cc:654] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:17.959089 8276 tablet_bootstrap.cc:654] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:17.959383 8274 log.cc:824] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:17.959677 8276 log.cc:824] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:17.959820 8275 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Bootstrap starting.
I20260811 10:10:17.960333 8274 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: No bootstrap required, opened a new log
I20260811 10:10:17.960410 8274 ts_tablet_manager.cc:1397] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:10:17.960937 8276 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: No bootstrap required, opened a new log
I20260811 10:10:17.961005 8276 ts_tablet_manager.cc:1397] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:10:17.962589 8274 raft_consensus.cc:348] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:17.962812 8274 raft_consensus.cc:374] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:17.962862 8274 raft_consensus.cc:729] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81663a104e7248f18ef16bcb25ebf08e, State: Initialized, Role: FOLLOWER
I20260811 10:10:17.963032 8274 consensus_queue.cc:260] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:17.963080 8276 raft_consensus.cc:348] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:17.963254 8276 raft_consensus.cc:374] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:17.963306 8276 raft_consensus.cc:729] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2389cdd77e2344a189a8b6e456aeee3e, State: Initialized, Role: FOLLOWER
I20260811 10:10:17.963392 8274 ts_tablet_manager.cc:1428] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:10:17.963425 8276 consensus_queue.cc:260] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:17.963445 8100 heartbeater.cc:503] Master 127.22.177.62:36473 was elected leader, sending a full tablet report...
I20260811 10:10:17.963702 8276 ts_tablet_manager.cc:1428] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:10:17.963757 7953 heartbeater.cc:503] Master 127.22.177.62:36473 was elected leader, sending a full tablet report...
I20260811 10:10:17.964558 8275 tablet_bootstrap.cc:654] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:17.965121 8275 log.cc:824] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:17.966005 8275 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: No bootstrap required, opened a new log
I20260811 10:10:17.966071 8275 ts_tablet_manager.cc:1397] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 10:10:17.967541 8275 raft_consensus.cc:348] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:17.967703 8275 raft_consensus.cc:374] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:17.967730 8275 raft_consensus.cc:729] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd45df631ba4211b9d9d902c53beefd, State: Initialized, Role: FOLLOWER
I20260811 10:10:17.967860 8275 consensus_queue.cc:260] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:17.968154 8275 ts_tablet_manager.cc:1428] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:10:17.968231 8251 heartbeater.cc:503] Master 127.22.177.62:36473 was elected leader, sending a full tablet report...
W20260811 10:10:17.968469 8101 tablet.cc:2369] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:10:18.027514 8282 raft_consensus.cc:482] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:18.027693 8282 raft_consensus.cc:504] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:18.028122 8282 leader_election.cc:288] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823), 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919)
I20260811 10:10:18.032003 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:18.028513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:52163 (local address 127.22.177.2:35823)
0811 10:10:18.028560 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:18.028562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:18.028662 (+ 100us) server_negotiation.cc:407] Connection header received
0811 10:10:18.028731 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:18.028732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:18.028743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:18.028821 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:18.028978 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.029529 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.030936 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.031578 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.031643 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:18.031650 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:18.031855 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 10:10:18.031899 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:10:18.032023 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:18.028404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35823 (local address 127.22.177.3:52163)
0811 10:10:18.028588 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:18.028597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:18.028664 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:18.028819 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:18.028820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:18.028821 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:18.028906 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:18.028908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.029712 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.029714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:18.030821 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:18.030824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.031691 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.031692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:18.031762 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:18.031767 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:10:18.031849 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 10:10:18.031894 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":134,"threads_started":1}
I20260811 10:10:18.032025 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:18.028427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:36815 (local address 127.22.177.1:36919)
0811 10:10:18.028477 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:18.028479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:18.028492 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:10:18.028531 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:18.028532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:18.028545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:18.028606 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:18.028789 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.029320 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.030937 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.031529 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.031638 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:18.031650 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:18.031889 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 10:10:18.031955 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:10:18.032025 8254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:18.028342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36919 (local address 127.22.177.3:36815)
0811 10:10:18.028409 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:18.028422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:18.028476 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:18.028612 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:18.028613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:18.028614 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:18.028702 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:18.028704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.029442 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.029443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:18.030821 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:18.030824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.031667 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.031669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:18.031774 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:18.031778 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:10:18.031878 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 10:10:18.031928 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":27}
I20260811 10:10:18.032488 8050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba97d3b33be44288a0afbf6fc39dcbb" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81663a104e7248f18ef16bcb25ebf08e" is_pre_election: true
I20260811 10:10:18.032688 8050 raft_consensus.cc:2454] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 0.
I20260811 10:10:18.032645 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba97d3b33be44288a0afbf6fc39dcbb" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2389cdd77e2344a189a8b6e456aeee3e" is_pre_election: true
I20260811 10:10:18.032850 7899 raft_consensus.cc:2454] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 0.
I20260811 10:10:18.033049 8127 leader_election.cc:302] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd45df631ba4211b9d9d902c53beefd, 81663a104e7248f18ef16bcb25ebf08e; no voters:
I20260811 10:10:18.033205 8282 raft_consensus.cc:2781] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:10:18.033262 8282 raft_consensus.cc:482] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:10:18.033284 8282 raft_consensus.cc:3037] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:18.034214 8282 raft_consensus.cc:504] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:18.034392 8282 leader_election.cc:288] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 election: Requested vote from peers 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823), 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919)
I20260811 10:10:18.034572 8050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba97d3b33be44288a0afbf6fc39dcbb" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81663a104e7248f18ef16bcb25ebf08e"
I20260811 10:10:18.034610 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba97d3b33be44288a0afbf6fc39dcbb" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2389cdd77e2344a189a8b6e456aeee3e"
I20260811 10:10:18.034690 7899 raft_consensus.cc:3037] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:18.034684 8050 raft_consensus.cc:3037] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:18.035674 7899 raft_consensus.cc:2454] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 1.
I20260811 10:10:18.035674 8050 raft_consensus.cc:2454] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 1.
I20260811 10:10:18.035969 8131 leader_election.cc:302] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd45df631ba4211b9d9d902c53beefd, 2389cdd77e2344a189a8b6e456aeee3e; no voters:
I20260811 10:10:18.036142 8282 raft_consensus.cc:2781] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:10:18.036371 8282 raft_consensus.cc:686] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 1 LEADER]: Becoming Leader. State: Replica: 1bd45df631ba4211b9d9d902c53beefd, State: Running, Role: LEADER
I20260811 10:10:18.036504 8282 consensus_queue.cc:237] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
W20260811 10:10:18.037622 7954 tablet.cc:2369] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:10:18.037490 7733 catalog_manager.cc:5495] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3). New cstate: current_term: 1 leader_uuid: "1bd45df631ba4211b9d9d902c53beefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } health_report { overall_health: HEALTHY } } }
I20260811 10:10:18.050494 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:18.047634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.22.177.3:45295)
0811 10:10:18.047724 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:18.047726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:18.047738 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:10:18.047801 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:18.047802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:18.047822 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:18.047898 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:10:18.048042 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.048599 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.049467 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:18.049754 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:18.049829 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:18.050046 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:10:18.050166 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:10:18.050279 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:18.050305 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 10:10:18.050330 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 10:10:18.052603 7899 raft_consensus.cc:1261] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Refusing update from remote peer 1bd45df631ba4211b9d9d902c53beefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:10:18.052994 8050 raft_consensus.cc:1261] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Refusing update from remote peer 1bd45df631ba4211b9d9d902c53beefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:10:18.053049 8284 consensus_queue.cc:1035] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:10:18.053421 8284 consensus_queue.cc:1035] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:10:18.056730 8291 mvcc.cc:205] Tried to move back new op lower bound from 7317270601940471808 to 7317270601878360064. Current Snapshot: MvccSnapshot[applied={T|T < 7317270601940471808}]
I20260811 10:10:18.059163 7734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:10:18.059309 7734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:10:18.060207 7734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Sending DeleteTablet for 3 replicas of tablet aba97d3b33be44288a0afbf6fc39dcbb
I20260811 10:10:18.060672 8029 tablet_service.cc:1514] Processing DeleteTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:44766
I20260811 10:10:18.060702 7875 tablet_service.cc:1514] Processing DeleteTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:35742
I20260811 10:10:18.060968 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 7803
I20260811 10:10:18.061005 8182 tablet_service.cc:1514] Processing DeleteTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:56244
I20260811 10:10:18.061007 8301 tablet_replica.cc:330] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: stopping tablet replica
I20260811 10:10:18.061350 8301 raft_consensus.cc:2229] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:10:18.061558 8301 raft_consensus.cc:2258] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:18.062768 8301 ts_tablet_manager.cc:1905] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:18.065167 8301 ts_tablet_manager.cc:1918] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:18.065186 8302 tablet_replica.cc:330] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: stopping tablet replica
I20260811 10:10:18.065340 8302 raft_consensus.cc:2229] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:10:18.065456 8301 log.cc:1196] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/aba97d3b33be44288a0afbf6fc39dcbb
I20260811 10:10:18.065521 8302 raft_consensus.cc:2258] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:18.066174 8301 ts_tablet_manager.cc:1939] T aba97d3b33be44288a0afbf6fc39dcbb P 81663a104e7248f18ef16bcb25ebf08e: Deleting consensus metadata
I20260811 10:10:18.066596 8302 ts_tablet_manager.cc:1905] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:18.067087 7715 catalog_manager.cc:4841] TS 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823): tablet aba97d3b33be44288a0afbf6fc39dcbb (table test-table [id=30bc459f66624f7883b903d743a130b1]) successfully deleted
I20260811 10:10:18.068859 8302 ts_tablet_manager.cc:1918] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:18.068962 8302 log.cc:1196] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/aba97d3b33be44288a0afbf6fc39dcbb
I20260811 10:10:18.069434 8302 ts_tablet_manager.cc:1939] T aba97d3b33be44288a0afbf6fc39dcbb P 1bd45df631ba4211b9d9d902c53beefd: Deleting consensus metadata
I20260811 10:10:18.070055 7719 catalog_manager.cc:4841] TS 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295): tablet aba97d3b33be44288a0afbf6fc39dcbb (table test-table [id=30bc459f66624f7883b903d743a130b1]) successfully deleted
W20260811 10:10:18.072453 7719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:36919 (error 108)
I20260811 10:10:18.072646 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 7958
I20260811 10:10:18.074700 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:18.074537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36919 (local address 127.0.0.1:35752)
0811 10:10:18.074606 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:18.074651 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 10:10:18.074836 7719 catalog_manager.cc:4584] TS 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aba97d3b33be44288a0afbf6fc39dcbb: Network error: Client connection negotiation failed: client connection to 127.22.177.1:36919: connect: Connection refused (error 111)
I20260811 10:10:18.083416 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 8105
I20260811 10:10:18.092020 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 7696
I20260811 10:10:38.102406 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36473
--webserver_interface=127.22.177.62
--webserver_port=45219
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:38.192359 8304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:38.192596 8304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:38.192620 8304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:38.193907 8304 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 10:10:38.193960 8304 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 10:10:38.193974 8304 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 10:10:38.193986 8304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:10:38.194000 8304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:38.194012 8304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:38.194025 8304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:10:38.194037 8304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:10:38.195700 8304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=45219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:38.196027 8304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:38.196307 8304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:38.206730 8312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:38.206875 8304 server_base.cc:1034] running on GCE node
W20260811 10:10:38.206777 8314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.206786 8311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:38.207262 8304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:38.207659 8304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:38.208879 8304 hybrid_clock.cc:648] HybridClock initialized: now 1786443038208867 us; error 36 us; skew 500 ppm
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:38.215977 8304 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:10:38.216441 8304 webserver.cc:460] Webserver started at http://127.22.177.62:45219/ using document root <none> and password file <none>
I20260811 10:10:38.216737 8304 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:38.216802 8304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:38.218487 8304 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:10:38.219368 8321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.219630 8304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:38.219766 8304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e801e94c32214343bc4c7d80cde52a67"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b6208ef8eeae709b4c608ded6f38b0f5"
server_key_iv: "4927699a2ecf8f612e75880f56992277"
server_key_version: "encryptionkey@0"
I20260811 10:10:38.220290 8304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:38.252629 8304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:38.255137 8304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:38.255393 8304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:38.260087 8304 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:36473
I20260811 10:10:38.260123 8383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:36473 every 8 connection(s)
I20260811 10:10:38.260614 8304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:10:38.262480 8384 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:10:38.263257 8384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap starting.
I20260811 10:10:38.270756 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8304
I20260811 10:10:38.271147 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:36919
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:36473
--webserver_port=35227
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:10:38.272511 8384 log.cc:824] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:38.274760 8384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:10:38.275141 8384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap complete.
I20260811 10:10:38.277727 8384 raft_consensus.cc:348] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:38.278018 8384 raft_consensus.cc:729] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e801e94c32214343bc4c7d80cde52a67, State: Initialized, Role: FOLLOWER
I20260811 10:10:38.278210 8384 consensus_queue.cc:260] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:38.278301 8384 raft_consensus.cc:388] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:10:38.278365 8384 raft_consensus.cc:482] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:10:38.278416 8384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:10:38.279526 8384 raft_consensus.cc:504] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:38.279664 8384 leader_election.cc:302] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e801e94c32214343bc4c7d80cde52a67; no voters:
I20260811 10:10:38.279906 8384 leader_election.cc:288] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:10:38.279950 8388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:10:38.280090 8388 raft_consensus.cc:686] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 LEADER]: Becoming Leader. State: Replica: e801e94c32214343bc4c7d80cde52a67, State: Running, Role: LEADER
I20260811 10:10:38.280189 8388 consensus_queue.cc:237] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:38.280220 8384 sys_catalog.cc:564] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:10:38.280501 8388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e801e94c32214343bc4c7d80cde52a67. Latest consensus state: current_term: 2 leader_uuid: "e801e94c32214343bc4c7d80cde52a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } } }
I20260811 10:10:38.280586 8388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:38.280504 8389 sys_catalog.cc:455] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e801e94c32214343bc4c7d80cde52a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } } }
I20260811 10:10:38.280853 8389 sys_catalog.cc:458] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:38.281279 8393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:10:38.282333 8393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:10:38.282491 8393 catalog_manager.cc:1259] Loaded cluster ID: 9e693eb8f00b4060a76f2cc2fb01e65e
I20260811 10:10:38.282644 8393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:10:38.284044 8393 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:10:38.284233 8393 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Loaded TSK: 0
I20260811 10:10:38.298202 8393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Generated new TSK 1
I20260811 10:10:38.298328 8393 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:10:38.362001 8386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:38.362411 8386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:38.362521 8386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:38.363854 8386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:38.363942 8386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:38.363993 8386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:10:38.365715 8386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:36919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=35227
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:38.366010 8386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:38.366271 8386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:38.372689 8414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.372699 8417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.372699 8415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:38.373446 8386 server_base.cc:1034] running on GCE node
I20260811 10:10:38.373700 8386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:38.373987 8386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:38.375164 8386 hybrid_clock.cc:648] HybridClock initialized: now 1786443038375138 us; error 44 us; skew 500 ppm
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:38.378047 8386 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:10:38.378572 8386 webserver.cc:460] Webserver started at http://127.22.177.1:35227/ using document root <none> and password file <none>
I20260811 10:10:38.378820 8386 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:38.378916 8386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:38.380510 8386 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:38.381378 8424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.381567 8386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:38.381699 8386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "2389cdd77e2344a189a8b6e456aeee3e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b264482a2a9e261163294aefee27657c"
server_key_iv: "e00c66a5d1a30941b39a18af98e406ea"
server_key_version: "encryptionkey@0"
I20260811 10:10:38.382100 8386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:38.396451 8386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:38.398989 8386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:38.399169 8386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:38.399438 8386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:38.399896 8431 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:10:38.401921 8386 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:10:38.401993 8386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:10:38.402042 8386 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:10:38.402896 8386 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:10:38.402956 8386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:10:38.403033 8431 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap starting.
I20260811 10:10:38.410717 8386 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:36919
I20260811 10:10:38.410755 8554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:36919 every 8 connection(s)
I20260811 10:10:38.411178 8386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:10:38.414314 8431 log.cc:824] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:38.417371 8431 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:10:38.417829 8431 tablet_bootstrap.cc:492] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap complete.
I20260811 10:10:38.418084 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8386
I20260811 10:10:38.418447 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:35823
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:36473
--webserver_port=35585
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:10:38.419201 8431 ts_tablet_manager.cc:1397] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
I20260811 10:10:38.420892 8431 raft_consensus.cc:348] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:38.421275 8431 raft_consensus.cc:729] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2389cdd77e2344a189a8b6e456aeee3e, State: Initialized, Role: FOLLOWER
I20260811 10:10:38.421497 8431 consensus_queue.cc:260] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } } peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } }
I20260811 10:10:38.421885 8431 ts_tablet_manager.cc:1428] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:10:38.431453 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.411974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:54839)
0811 10:10:38.413811 (+ 1837us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.413862 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.414388 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.417729 (+ 3341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.417743 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.418196 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:38.418513 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.418521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.420575 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.420577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.422121 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.422127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.423027 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.423028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.423204 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.423939 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:38.423951 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:38.425945 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:38.428687 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.428699 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.428702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.429077 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.429576 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.429578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.429578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.429919 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.430591 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:38.430600 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:38.430773 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.430920 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.431117 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1761,"thread_start_us":98,"threads_started":1}
I20260811 10:10:38.432243 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.412048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:54839 (local address 127.22.177.62:36473)
0811 10:10:38.417132 (+ 5084us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.417140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.417180 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:10:38.417365 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.417371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.417420 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.417663 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:38.418689 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.420279 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.422269 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.422915 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.423045 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.426101 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:38.426108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:38.426108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:38.426123 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:38.428549 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.429200 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.429204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.429205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.429467 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.430038 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.430039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.430040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.430499 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:38.430607 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.431013 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.431194 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5022,"thread_start_us":85,"threads_started":1}
I20260811 10:10:38.433277 8556 heartbeater.cc:347] Connected to a master server at 127.22.177.62:36473
I20260811 10:10:38.433384 8556 heartbeater.cc:464] Registering TS with master...
I20260811 10:10:38.433657 8556 heartbeater.cc:511] Master 127.22.177.62:36473 requested a full tablet report, sending...
I20260811 10:10:38.434521 8340 ts_manager.cc:194] Registered new tserver with Master: 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919)
I20260811 10:10:38.435796 8340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:54839
I20260811 10:10:38.436772 8556 heartbeater.cc:503] Master 127.22.177.62:36473 was elected leader, sending a full tablet report...
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.1@KRBTEST.COM
I20260811 10:10:38.444892 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.435268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36919 (local address 127.0.0.1:60362)
0811 10:10:38.435477 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.435515 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.435575 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.435853 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.435859 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.436304 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:38.436371 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.436374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.437883 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.437884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.438494 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.438497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.439158 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.439159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.439224 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.439822 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:38.439826 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:38.441501 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:38.443651 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.443659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.443661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.443905 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.444147 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.444148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.444149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.444227 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.444567 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:38.444574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:38.444624 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.444694 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.444809 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":118,"threads_started":1}
I20260811 10:10:38.444983 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.435389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.22.177.1:36919)
0811 10:10:38.435699 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.435702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.435709 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:10:38.435737 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.435739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.435750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.435847 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:38.436458 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.437766 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.438580 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.439070 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.439142 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.441625 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:38.441638 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:38.441638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:38.441647 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:38.443556 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.443988 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.443992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.443992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.444058 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.444298 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.444299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.444299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.444496 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:38.444580 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.444761 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.444874 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":83,"threads_started":1}
I20260811 10:10:38.445386 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet aba97d3b33be44288a0afbf6fc39dcbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:18 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:60362
I20260811 10:10:38.446209 8565 tablet_replica.cc:330] stopping tablet replica
I20260811 10:10:38.446317 8565 raft_consensus.cc:2229] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:10:38.446441 8565 raft_consensus.cc:2258] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:38.446774 8565 ts_tablet_manager.cc:1905] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:38.449373 8565 ts_tablet_manager.cc:1918] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:38.449445 8565 log.cc:1196] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal/wals/aba97d3b33be44288a0afbf6fc39dcbb
I20260811 10:10:38.449741 8565 ts_tablet_manager.cc:1939] T aba97d3b33be44288a0afbf6fc39dcbb P 2389cdd77e2344a189a8b6e456aeee3e: Deleting consensus metadata
I20260811 10:10:38.450325 8325 catalog_manager.cc:4841] TS 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919): tablet aba97d3b33be44288a0afbf6fc39dcbb (table test-table [id=30bc459f66624f7883b903d743a130b1]) successfully deleted
W20260811 10:10:38.514461 8561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:38.514746 8561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:38.514789 8561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:38.516042 8561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:38.516122 8561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:38.516170 8561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:10:38.517959 8561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:35823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=35585
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:38.518260 8561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:38.518522 8561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:38.524696 8571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.524678 8575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.524677 8572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:38.524988 8561 server_base.cc:1034] running on GCE node
I20260811 10:10:38.525213 8561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:38.525447 8561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:38.526614 8561 hybrid_clock.cc:648] HybridClock initialized: now 1786443038526580 us; error 76 us; skew 500 ppm
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:38.530185 8561 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:10:38.531096 8561 webserver.cc:460] Webserver started at http://127.22.177.2:35585/ using document root <none> and password file <none>
I20260811 10:10:38.531438 8561 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:38.531528 8561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:38.533181 8561 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:38.534163 8585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.534349 8561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:10:38.534451 8561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "81663a104e7248f18ef16bcb25ebf08e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "7e7aa53d365e276690aabaf689ac1546"
server_key_iv: "3bfb97b9ea2fc8f9ae8e4ad502f884df"
server_key_version: "encryptionkey@0"
I20260811 10:10:38.534823 8561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:38.543952 8561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:38.546597 8561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:38.546775 8561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:38.547012 8561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:38.547413 8561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:38.547453 8561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.547478 8561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:38.547492 8561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.554507 8561 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35823
I20260811 10:10:38.554586 8720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35823 every 8 connection(s)
I20260811 10:10:38.554983 8561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:10:38.557698 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8561
I20260811 10:10:38.558063 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:45295
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:36473
--webserver_port=38361
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:10:38.568120 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.555970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:34621 (local address 127.22.177.62:36473)
0811 10:10:38.556048 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.556050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.556227 (+ 177us) server_negotiation.cc:407] Connection header received
0811 10:10:38.556911 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.556912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.556928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.556999 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:38.558499 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.559295 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.560994 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.561227 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.561299 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.564227 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:38.564229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:38.564229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:38.564239 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:38.566127 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.566847 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.566849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.566849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.566927 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.567253 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.567254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.567254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.567396 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:38.567480 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.567864 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.568011 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:10:38.568504 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.555882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:34621)
0811 10:10:38.556060 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.556104 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.556716 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.557146 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.557150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.557990 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:38.558373 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.558380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.559439 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.559440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.560870 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.560874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.561334 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.561336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.561498 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.562180 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:38.562191 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:38.564093 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:38.566396 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.566409 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.566413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.566744 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.567016 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.567017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.567018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.567158 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.567480 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:38.567491 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:38.567685 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.567827 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.568031 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260811 10:10:38.569080 8721 heartbeater.cc:347] Connected to a master server at 127.22.177.62:36473
I20260811 10:10:38.569224 8721 heartbeater.cc:464] Registering TS with master...
I20260811 10:10:38.569586 8721 heartbeater.cc:511] Master 127.22.177.62:36473 requested a full tablet report, sending...
I20260811 10:10:38.570137 8340 ts_manager.cc:194] Registered new tserver with Master: 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823)
I20260811 10:10:38.570581 8340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:34621
W20260811 10:10:38.652513 8724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:38.652755 8724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:38.652782 8724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:38.654111 8724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:38.654165 8724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:38.654194 8724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:10:38.655772 8724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:45295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=38361
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:38.656066 8724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:38.656353 8724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:38.662097 8729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.662091 8730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:38.662127 8732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:38.662421 8724 server_base.cc:1034] running on GCE node
I20260811 10:10:38.662659 8724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:38.662926 8724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:38.664110 8724 hybrid_clock.cc:648] HybridClock initialized: now 1786443038664106 us; error 40 us; skew 500 ppm
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:38.666972 8724 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:10:38.667569 8724 webserver.cc:460] Webserver started at http://127.22.177.3:38361/ using document root <none> and password file <none>
I20260811 10:10:38.667857 8724 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:38.667948 8724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:38.669440 8724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:10:38.670202 8740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.670429 8724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:10:38.670542 8724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "1bd45df631ba4211b9d9d902c53beefd"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "dbf46e03a9d30820bd92e8581192a414"
server_key_iv: "88029766e602d82568d361c693e1af19"
server_key_version: "encryptionkey@0"
I20260811 10:10:38.670930 8724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:38.691979 8724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:38.694460 8724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:38.694679 8724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:38.694957 8724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:38.695346 8724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:38.695410 8724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.695462 8724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:38.695497 8724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:38.702301 8724 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:45295
I20260811 10:10:38.702458 8867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:45295 every 8 connection(s)
I20260811 10:10:38.702790 8724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:10:38.706236 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8724
Aug 11 10:10:38 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:10:38.714896 8872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.706784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.22.177.62:36473)
0811 10:10:38.706918 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.706921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.707032 (+ 111us) server_negotiation.cc:407] Connection header received
0811 10:10:38.707109 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.707110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.707127 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.707194 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:38.708040 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.708517 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.709612 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.709927 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.710025 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.711483 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:38.711484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:38.711484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:38.711490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:38.713385 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.713928 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.713930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.713930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.713987 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.714278 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.714279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.714280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.714436 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:38.714641 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.714691 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.714783 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260811 10:10:38.715299 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.703651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:56469)
0811 10:10:38.703842 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.703882 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.704469 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.704786 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.704790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.705454 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:10:38.705703 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.705708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.706786 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.706788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.708261 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.708264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.708735 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.708737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.708852 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.709469 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:10:38.709478 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:10:38.711151 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:10:38.713219 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.713227 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.713230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.713556 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.713905 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:10:38.713907 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:10:38.713907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:10:38.714073 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:10:38.714497 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:10:38.714504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:10:38.714690 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.714801 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.715072 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260811 10:10:38.715554 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.703687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:56469 (local address 127.22.177.62:36473)
0811 10:10:38.703750 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.703753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.703984 (+ 231us) server_negotiation.cc:407] Connection header received
0811 10:10:38.704598 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.704599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.704613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.704676 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:38.705833 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.706689 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.708357 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.708640 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.708718 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.711256 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:38.711258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:38.711258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:38.711267 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:38.713121 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.713708 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.713710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.713710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.713781 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.714189 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.714190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.714191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.714423 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:38.714481 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.714880 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.714978 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:10:38.715930 8868 heartbeater.cc:347] Connected to a master server at 127.22.177.62:36473
I20260811 10:10:38.716068 8868 heartbeater.cc:464] Registering TS with master...
I20260811 10:10:38.716264 8868 heartbeater.cc:511] Master 127.22.177.62:36473 requested a full tablet report, sending...
I20260811 10:10:38.716646 8340 ts_manager.cc:194] Registered new tserver with Master: 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295)
I20260811 10:10:38.717027 8340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:56469
I20260811 10:10:38.727262 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:10:38.730882 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.727804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.22.177.62:36473)
0811 10:10:38.727873 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.727876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.727981 (+ 105us) server_negotiation.cc:407] Connection header received
0811 10:10:38.728068 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.728068 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.728109 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.728181 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:10:38.728403 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.728973 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.729870 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.730121 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.730209 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.730488 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:10:38.730542 (+ 54us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 10:10:38.730772 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36682: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 10:10:38.730959 8559 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36682: token expired
W20260811 10:10:38.731148 8256 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.177.62:36473)
I20260811 10:10:38.738692 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.731505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.22.177.62:36473)
0811 10:10:38.731579 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.731580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.731588 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:10:38.731605 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.731606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.731614 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.731661 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:10:38.732274 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.732753 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.733579 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.733935 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.734000 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.735579 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:10:38.735580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:10:38.735580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:10:38.735588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:10:38.737154 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.737567 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.737568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.737568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.737710 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:10:38.737997 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:10:38.737998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:10:38.737999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:10:38.738128 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:10:38.738212 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.738467 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.738587 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 10:10:38.740136 8340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:10:38.741951 8340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:10:38.744653 8461 tablet_service.cc:1467] Processing CreateTablet for tablet c643a4fa88a441668d16dd58d97643b3 (DEFAULT_TABLE table=test-table [id=9ede6748e469489ba3a2b5e9e1263fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:10:38.744917 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c643a4fa88a441668d16dd58d97643b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:38.746373 8431 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap starting.
I20260811 10:10:38.746845 8431 tablet_bootstrap.cc:654] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:38.747573 8431 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: No bootstrap required, opened a new log
I20260811 10:10:38.747639 8431 ts_tablet_manager.cc:1397] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:10:38.748046 8431 raft_consensus.cc:348] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.748149 8431 raft_consensus.cc:374] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:38.748216 8431 raft_consensus.cc:729] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2389cdd77e2344a189a8b6e456aeee3e, State: Initialized, Role: FOLLOWER
I20260811 10:10:38.748271 8431 consensus_queue.cc:260] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.748381 8431 ts_tablet_manager.cc:1428] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 10:10:38.749311 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.744132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35823 (local address 127.0.0.1:58290)
0811 10:10:38.744227 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.744240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.744286 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.744742 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.744743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.744745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:38.744818 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.744820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.746470 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.746471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.747904 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.747910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.749063 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.749064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.749138 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.749141 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.749206 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.749247 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":54}
I20260811 10:10:38.749310 8874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.744154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45295 (local address 127.0.0.1:38898)
0811 10:10:38.744370 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.744380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.744416 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.744831 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.744831 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.744834 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:38.744881 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.744883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.746444 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.746445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.747864 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.747868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.748932 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.748933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.749014 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.749044 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.749102 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.749196 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":75,"thread_start_us":168,"threads_started":1}
I20260811 10:10:38.749533 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.744283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38898 (local address 127.22.177.3:45295)
0811 10:10:38.744533 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.744539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.744552 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:10:38.744600 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.744602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.744622 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.744726 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:38.744974 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.746325 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.747961 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.748726 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.748851 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.748922 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.749371 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.749427 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
I20260811 10:10:38.749819 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.744231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58290 (local address 127.22.177.2:35823)
0811 10:10:38.744533 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.744539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.744552 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:10:38.744600 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.744603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.744622 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.744726 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:38.745009 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.746328 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.748043 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.748956 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.749541 (+ 585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.749646 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.749679 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.749740 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":106,"threads_started":1}
I20260811 10:10:38.750403 8796 tablet_service.cc:1467] Processing CreateTablet for tablet c643a4fa88a441668d16dd58d97643b3 (DEFAULT_TABLE table=test-table [id=9ede6748e469489ba3a2b5e9e1263fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:10:38.750418 8644 tablet_service.cc:1467] Processing CreateTablet for tablet c643a4fa88a441668d16dd58d97643b3 (DEFAULT_TABLE table=test-table [id=9ede6748e469489ba3a2b5e9e1263fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:10:38.750782 8796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c643a4fa88a441668d16dd58d97643b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:38.750811 8644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c643a4fa88a441668d16dd58d97643b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:10:38.753774 8878 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Bootstrap starting.
I20260811 10:10:38.754144 8879 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Bootstrap starting.
I20260811 10:10:38.754530 8878 tablet_bootstrap.cc:654] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:38.754920 8879 tablet_bootstrap.cc:654] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Neither blocks nor log segments found. Creating new log.
I20260811 10:10:38.755179 8878 log.cc:824] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:38.755606 8879 log.cc:824] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:38.756209 8878 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: No bootstrap required, opened a new log
I20260811 10:10:38.756306 8878 ts_tablet_manager.cc:1397] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:10:38.756592 8879 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: No bootstrap required, opened a new log
I20260811 10:10:38.756654 8879 ts_tablet_manager.cc:1397] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:10:38.758070 8878 raft_consensus.cc:348] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.758270 8878 raft_consensus.cc:374] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:38.758315 8878 raft_consensus.cc:729] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd45df631ba4211b9d9d902c53beefd, State: Initialized, Role: FOLLOWER
I20260811 10:10:38.758467 8878 consensus_queue.cc:260] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.758481 8879 raft_consensus.cc:348] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.758634 8879 raft_consensus.cc:374] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:10:38.758697 8879 raft_consensus.cc:729] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81663a104e7248f18ef16bcb25ebf08e, State: Initialized, Role: FOLLOWER
I20260811 10:10:38.758795 8878 ts_tablet_manager.cc:1428] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:10:38.758874 8879 consensus_queue.cc:260] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.758939 8868 heartbeater.cc:503] Master 127.22.177.62:36473 was elected leader, sending a full tablet report...
I20260811 10:10:38.759215 8879 ts_tablet_manager.cc:1428] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:10:38.759229 8721 heartbeater.cc:503] Master 127.22.177.62:36473 was elected leader, sending a full tablet report...
W20260811 10:10:38.805855 8722 tablet.cc:2369] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:10:38.824075 8885 raft_consensus.cc:482] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:10:38.824242 8885 raft_consensus.cc:504] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.824827 8885 leader_election.cc:288] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823), 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919)
I20260811 10:10:38.827943 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.825047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36919 (local address 127.22.177.3:33787)
0811 10:10:38.825109 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.825122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.825177 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.825416 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.825417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.825420 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:38.825494 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.825496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.826228 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.826229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.826987 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.826989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.827634 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.827635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.827724 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.827729 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.827819 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.827862 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:10:38.827953 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.825180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:33787 (local address 127.22.177.1:36919)
0811 10:10:38.825232 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.825234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.825242 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:10:38.825286 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.825287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.825316 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.825409 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:38.825570 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.826131 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.827090 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.827541 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.827634 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.827721 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.827826 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.827871 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:10:38.828269 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.825228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:60665 (local address 127.22.177.2:35823)
0811 10:10:38.825283 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.825286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.825467 (+ 181us) server_negotiation.cc:407] Connection header received
0811 10:10:38.825546 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.825547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.825561 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.825694 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:10:38.825876 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.826376 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.827419 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.827848 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.827944 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.827954 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.828144 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.828199 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:10:38.828299 8498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c643a4fa88a441668d16dd58d97643b3" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2389cdd77e2344a189a8b6e456aeee3e" is_pre_election: true
I20260811 10:10:38.828521 8498 raft_consensus.cc:2454] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 0.
I20260811 10:10:38.828560 8887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.825067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35823 (local address 127.22.177.3:60665)
0811 10:10:38.825324 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.825383 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:38.825479 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:38.825703 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:10:38.825704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:10:38.825706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:10:38.825788 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.825789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.826469 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.826470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.827305 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:10:38.827307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.827944 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.827946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:10:38.828033 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.828038 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:10:38.828411 (+ 373us) client_negotiation.cc:240] Negotiation successful
0811 10:10:38.828469 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":19,"thread_start_us":137,"threads_started":1}
I20260811 10:10:38.828841 8744 leader_election.cc:302] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd45df631ba4211b9d9d902c53beefd, 2389cdd77e2344a189a8b6e456aeee3e; no voters:
I20260811 10:10:38.829005 8885 raft_consensus.cc:2781] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:10:38.828975 8667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c643a4fa88a441668d16dd58d97643b3" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81663a104e7248f18ef16bcb25ebf08e" is_pre_election: true
I20260811 10:10:38.829083 8885 raft_consensus.cc:482] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:10:38.829114 8885 raft_consensus.cc:3037] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:38.829150 8667 raft_consensus.cc:2454] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 0.
I20260811 10:10:38.830025 8885 raft_consensus.cc:504] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.830214 8885 leader_election.cc:288] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 election: Requested vote from peers 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823), 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919)
I20260811 10:10:38.830439 8667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c643a4fa88a441668d16dd58d97643b3" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81663a104e7248f18ef16bcb25ebf08e"
I20260811 10:10:38.830513 8667 raft_consensus.cc:3037] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:38.830490 8498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c643a4fa88a441668d16dd58d97643b3" candidate_uuid: "1bd45df631ba4211b9d9d902c53beefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2389cdd77e2344a189a8b6e456aeee3e"
I20260811 10:10:38.830574 8498 raft_consensus.cc:3037] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:10:38.831204 8667 raft_consensus.cc:2454] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 1.
I20260811 10:10:38.831400 8498 raft_consensus.cc:2454] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bd45df631ba4211b9d9d902c53beefd in term 1.
I20260811 10:10:38.831434 8741 leader_election.cc:302] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd45df631ba4211b9d9d902c53beefd, 81663a104e7248f18ef16bcb25ebf08e; no voters:
I20260811 10:10:38.831559 8885 raft_consensus.cc:2781] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:10:38.831737 8885 raft_consensus.cc:686] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 1 LEADER]: Becoming Leader. State: Replica: 1bd45df631ba4211b9d9d902c53beefd, State: Running, Role: LEADER
I20260811 10:10:38.831892 8885 consensus_queue.cc:237] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:38.832921 8340 catalog_manager.cc:5495] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3). New cstate: current_term: 1 leader_uuid: "1bd45df631ba4211b9d9d902c53beefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } health_report { overall_health: UNKNOWN } } }
I20260811 10:10:38.844585 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.841977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.22.177.3:45295)
0811 10:10:38.842059 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:10:38.842062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:10:38.842108 (+ 46us) server_negotiation.cc:407] Connection header received
0811 10:10:38.842250 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:10:38.842251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:10:38.842269 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:10:38.842356 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:10:38.842490 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.842965 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.843745 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:10:38.843947 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:10:38.844031 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:10:38.844195 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:10:38.844301 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:10:38.844393 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:10:38.844435 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 10:10:38.844459 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:10:38.846756 8667 raft_consensus.cc:1261] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Refusing update from remote peer 1bd45df631ba4211b9d9d902c53beefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:10:38.846800 8498 raft_consensus.cc:1261] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Refusing update from remote peer 1bd45df631ba4211b9d9d902c53beefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:10:38.847254 8888 consensus_queue.cc:1035] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:10:38.847422 8885 consensus_queue.cc:1035] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:10:38.850370 8891 mvcc.cc:205] Tried to move back new op lower bound from 7317270687113498624 to 7317270687056175104. Current Snapshot: MvccSnapshot[applied={T|T < 7317270687113498624}]
I20260811 10:10:38.850355 8892 mvcc.cc:205] Tried to move back new op lower bound from 7317270687113498624 to 7317270687056175104. Current Snapshot: MvccSnapshot[applied={T|T < 7317270687113498624}]
I20260811 10:10:38.853672 8338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:10:38.853812 8338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:10:38.854821 8338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Sending DeleteTablet for 3 replicas of tablet c643a4fa88a441668d16dd58d97643b3
I20260811 10:10:38.855284 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet c643a4fa88a441668d16dd58d97643b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:38 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:60362
I20260811 10:10:38.855309 8796 tablet_service.cc:1514] Processing DeleteTablet for tablet c643a4fa88a441668d16dd58d97643b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:38 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:38898
I20260811 10:10:38.855492 8565 tablet_replica.cc:330] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: stopping tablet replica
I20260811 10:10:38.855551 8565 raft_consensus.cc:2229] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:10:38.855629 8565 raft_consensus.cc:2258] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:38.855684 8901 tablet_replica.cc:330] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: stopping tablet replica
I20260811 10:10:38.855811 8901 raft_consensus.cc:2229] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:10:38.855788 8644 tablet_service.cc:1514] Processing DeleteTablet for tablet c643a4fa88a441668d16dd58d97643b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:10:38 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:58290
I20260811 10:10:38.855983 8901 raft_consensus.cc:2258] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:38.856335 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 8386
I20260811 10:10:38.856498 8901 ts_tablet_manager.cc:1905] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:38.858618 8902 tablet_replica.cc:330] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: stopping tablet replica
I20260811 10:10:38.858850 8902 raft_consensus.cc:2229] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:10:38.859053 8902 raft_consensus.cc:2258] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:10:38.859663 8902 ts_tablet_manager.cc:1905] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:10:38.859817 8901 ts_tablet_manager.cc:1918] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:38.859916 8901 log.cc:1196] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/c643a4fa88a441668d16dd58d97643b3
I20260811 10:10:38.860294 8901 ts_tablet_manager.cc:1939] T c643a4fa88a441668d16dd58d97643b3 P 1bd45df631ba4211b9d9d902c53beefd: Deleting consensus metadata
I20260811 10:10:38.860809 8325 catalog_manager.cc:4841] TS 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295): tablet c643a4fa88a441668d16dd58d97643b3 (table test-table [id=9ede6748e469489ba3a2b5e9e1263fe9]) successfully deleted
I20260811 10:10:38.862396 8902 ts_tablet_manager.cc:1918] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:10:38.862533 8902 log.cc:1196] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/c643a4fa88a441668d16dd58d97643b3
I20260811 10:10:38.862875 8902 ts_tablet_manager.cc:1939] T c643a4fa88a441668d16dd58d97643b3 P 81663a104e7248f18ef16bcb25ebf08e: Deleting consensus metadata
I20260811 10:10:38.863463 8322 catalog_manager.cc:4841] TS 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823): tablet c643a4fa88a441668d16dd58d97643b3 (table test-table [id=9ede6748e469489ba3a2b5e9e1263fe9]) successfully deleted
W20260811 10:10:38.866298 8325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:36919 (error 108)
I20260811 10:10:38.866662 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 8561
I20260811 10:10:38.866885 8874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:38.866747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:36919 (local address 127.0.0.1:60366)
0811 10:10:38.866809 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:38.866846 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 10:10:38.867015 8325 catalog_manager.cc:4584] TS 2389cdd77e2344a189a8b6e456aeee3e (127.22.177.1:36919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c643a4fa88a441668d16dd58d97643b3: Network error: Client connection negotiation failed: client connection to 127.22.177.1:36919: connect: Connection refused (error 111)
I20260811 10:10:38.876904 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 8724
I20260811 10:10:38.887048 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 8304
I20260811 10:10:58.897774 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36473
--webserver_interface=127.22.177.62
--webserver_port=45219
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:59.004878 8904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:59.005132 8904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:59.005158 8904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:59.006567 8904 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 10:10:59.006625 8904 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 10:10:59.006639 8904 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 10:10:59.006650 8904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:10:59.006664 8904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:59.006677 8904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:59.006732 8904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:10:59.006752 8904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:10:59.008466 8904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=45219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:59.008795 8904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:59.009063 8904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:59.014827 8912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.014870 8909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.014827 8910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:59.015363 8904 server_base.cc:1034] running on GCE node
I20260811 10:10:59.015666 8904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:59.016017 8904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:59.017189 8904 hybrid_clock.cc:648] HybridClock initialized: now 1786443059017155 us; error 67 us; skew 500 ppm
Aug 11 10:10:59 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443059, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:59.020401 8904 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:10:59.021013 8904 webserver.cc:460] Webserver started at http://127.22.177.62:45219/ using document root <none> and password file <none>
I20260811 10:10:59.021304 8904 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:59.021358 8904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:59.023048 8904 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:59.024050 8923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.024541 8904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:59.024691 8904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e801e94c32214343bc4c7d80cde52a67"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b6208ef8eeae709b4c608ded6f38b0f5"
server_key_iv: "4927699a2ecf8f612e75880f56992277"
server_key_version: "encryptionkey@0"
I20260811 10:10:59.025175 8904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:59.040267 8904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:59.043186 8904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:59.043452 8904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:59.048112 8904 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:36473
I20260811 10:10:59.048151 8980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:36473 every 8 connection(s)
I20260811 10:10:59.048604 8904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:10:59.050419 8981 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:10:59.051165 8981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap starting.
I20260811 10:10:59.056082 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8904
I20260811 10:10:59.056479 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:36919
--local_ip_for_outbound_sockets=127.22.177.1
--tserver_master_addrs=127.22.177.62:36473
--webserver_port=35227
--webserver_interface=127.22.177.1
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:10:59.061975 8981 log.cc:824] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:59.064435 8981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:10:59.064837 8981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Bootstrap complete.
I20260811 10:10:59.067492 8981 raft_consensus.cc:348] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:59.067770 8981 raft_consensus.cc:729] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e801e94c32214343bc4c7d80cde52a67, State: Initialized, Role: FOLLOWER
I20260811 10:10:59.068109 8981 consensus_queue.cc:260] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:59.068226 8981 raft_consensus.cc:388] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:10:59.068262 8981 raft_consensus.cc:482] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:10:59.068356 8981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 2 FOLLOWER]: Advancing to term 3
I20260811 10:10:59.069677 8981 raft_consensus.cc:504] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:59.069792 8981 leader_election.cc:302] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e801e94c32214343bc4c7d80cde52a67; no voters:
I20260811 10:10:59.070060 8981 leader_election.cc:288] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 10:10:59.070101 8987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 10:10:59.070230 8987 raft_consensus.cc:686] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [term 3 LEADER]: Becoming Leader. State: Replica: e801e94c32214343bc4c7d80cde52a67, State: Running, Role: LEADER
I20260811 10:10:59.070338 8987 consensus_queue.cc:237] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } }
I20260811 10:10:59.070416 8981 sys_catalog.cc:564] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:10:59.070633 8987 sys_catalog.cc:455] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e801e94c32214343bc4c7d80cde52a67. Latest consensus state: current_term: 3 leader_uuid: "e801e94c32214343bc4c7d80cde52a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } } }
I20260811 10:10:59.070714 8987 sys_catalog.cc:458] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:59.070637 8989 sys_catalog.cc:455] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e801e94c32214343bc4c7d80cde52a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e801e94c32214343bc4c7d80cde52a67" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 36473 } } }
I20260811 10:10:59.070927 8989 sys_catalog.cc:458] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67 [sys.catalog]: This master's current role is: LEADER
I20260811 10:10:59.070945 8992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:10:59.072037 8992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:10:59.072155 8992 catalog_manager.cc:1259] Loaded cluster ID: 9e693eb8f00b4060a76f2cc2fb01e65e
I20260811 10:10:59.072194 8992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:10:59.073442 8992 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:10:59.073709 8992 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Loaded TSK: 0
I20260811 10:10:59.073769 8992 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e801e94c32214343bc4c7d80cde52a67: Loaded TSK: 1
F20260811 10:10:59.074045 8992 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa4538859ad google::LogMessage::Fail() at ??:0
@ 0x7fa45388888a google::LogMessage::SendToLog() at ??:0
@ 0x7fa4538854de google::LogMessage::Flush() at ??:0
@ 0x7fa453886a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa455ff2104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa455fefd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa45b7e31cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa45b7e348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa45b7e3594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa45b7a9c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa45b7b3ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa4557530d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa455748954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa4565ac6db start_thread at ??:0
@ 0x7fa45238361f clone at ??:0
W20260811 10:10:59.184247 8983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:59.184603 8983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:59.184650 8983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:59.186991 8983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:59.187093 8983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:59.187145 8983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:10:59.190016 8983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:36919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=35227
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:59.190371 8983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:59.190682 8983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:59.197088 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.197075 9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.197077 9032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:59.197702 8983 server_base.cc:1034] running on GCE node
I20260811 10:10:59.197953 8983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:59.198230 8983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:59.199455 8983 hybrid_clock.cc:648] HybridClock initialized: now 1786443059199439 us; error 33 us; skew 500 ppm
Aug 11 10:10:59 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443059, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:59.202708 8983 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:10:59.203222 8983 webserver.cc:460] Webserver started at http://127.22.177.1:35227/ using document root <none> and password file <none>
I20260811 10:10:59.203509 8983 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:59.203589 8983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:59.205264 8983 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:10:59.206228 9039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.206504 8983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:10:59.206627 8983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "2389cdd77e2344a189a8b6e456aeee3e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "b264482a2a9e261163294aefee27657c"
server_key_iv: "e00c66a5d1a30941b39a18af98e406ea"
server_key_version: "encryptionkey@0"
I20260811 10:10:59.207058 8983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:59.224290 8983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:59.226867 8983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:59.227061 8983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:59.227339 8983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:59.227872 9047 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:10:59.230222 8983 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:10:59.230398 8983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260811 10:10:59.230445 8983 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:10:59.231244 8983 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:10:59.231319 8983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:10:59.231460 9047 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap starting.
I20260811 10:10:59.240185 8983 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:36919
I20260811 10:10:59.240243 9168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:36919 every 8 connection(s)
I20260811 10:10:59.240794 8983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:10:59.244414 9047 log.cc:824] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:10:59.246779 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 8983
I20260811 10:10:59.247164 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:35823
--local_ip_for_outbound_sockets=127.22.177.2
--tserver_master_addrs=127.22.177.62:36473
--webserver_port=35585
--webserver_interface=127.22.177.2
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:10:59.247866 9047 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:10:59.248219 9047 tablet_bootstrap.cc:492] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Bootstrap complete.
I20260811 10:10:59.249678 9047 ts_tablet_manager.cc:1397] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.010s
I20260811 10:10:59.251477 9047 raft_consensus.cc:348] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:59.251868 9047 raft_consensus.cc:729] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2389cdd77e2344a189a8b6e456aeee3e, State: Initialized, Role: FOLLOWER
I20260811 10:10:59.252064 9047 consensus_queue.cc:260] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:10:59.252398 9047 ts_tablet_manager.cc:1428] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20260811 10:10:59.336200 9171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:10:59.241744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:55595)
0811 10:10:59.241937 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.241984 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:10:59.242477 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:10:59.335984 (+ 93507us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
W20260811 10:10:59.336544 9169 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:36473 (0 consecutive failures): Network error: Failed to ping master at 127.22.177.62:36473: Client connection negotiation failed: client connection to 127.22.177.62:36473: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:10:59.336982 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.336848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:55995)
0811 10:10:59.336909 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.336962 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:10:59.337317 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.337228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:52825)
0811 10:10:59.337274 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.337297 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 10:10:59.337479 9169 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:10:59.351468 9173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:59.351790 9173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:59.351841 9173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:59.353245 9173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:59.353334 9173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:59.353389 9173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:10:59.355417 9173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:35823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=35585
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:59.355765 9173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:59.356073 9173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:59.361970 9179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.362013 9181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.362052 9183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:59.362725 9173 server_base.cc:1034] running on GCE node
I20260811 10:10:59.363027 9173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:59.363351 9173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:59.364529 9173 hybrid_clock.cc:648] HybridClock initialized: now 1786443059364523 us; error 39 us; skew 500 ppm
Aug 11 10:10:59 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443059, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:59.367533 9173 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:10:59.368139 9173 webserver.cc:460] Webserver started at http://127.22.177.2:35585/ using document root <none> and password file <none>
I20260811 10:10:59.368404 9173 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:59.368479 9173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:59.370033 9173 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:10:59.370772 9191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.371003 9173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:10:59.371124 9173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "81663a104e7248f18ef16bcb25ebf08e"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "7e7aa53d365e276690aabaf689ac1546"
server_key_iv: "3bfb97b9ea2fc8f9ae8e4ad502f884df"
server_key_version: "encryptionkey@0"
I20260811 10:10:59.371503 9173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:59.388823 9173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:59.391697 9173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:59.391908 9173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:59.392185 9173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:59.392545 9173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:59.392604 9173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.392686 9173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:59.392719 9173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.400652 9173 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35823
I20260811 10:10:59.400749 9325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35823 every 8 connection(s)
I20260811 10:10:59.401211 9173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:10:59.402539 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.402075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:60533)
0811 10:10:59.402305 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.402387 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
W20260811 10:10:59.403057 9326 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:36473 (0 consecutive failures): Network error: Failed to ping master at 127.22.177.62:36473: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
I20260811 10:10:59.403467 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.403322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:53813)
0811 10:10:59.403407 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.403443 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 10:10:59.403913 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.403804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:40463)
0811 10:10:59.403859 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.403887 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 10:10:59.404027 9326 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:10:59.404706 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 9173
I20260811 10:10:59.405052 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:45295
--local_ip_for_outbound_sockets=127.22.177.3
--tserver_master_addrs=127.22.177.62:36473
--webserver_port=38361
--webserver_interface=127.22.177.3
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:10:59.495324 9330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:10:59.495563 9330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:10:59.495591 9330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:10:59.496909 9330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:10:59.496966 9330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:10:59.496996 9330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:10:59.498618 9330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:45295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=38361
--tserver_master_addrs=127.22.177.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:10:59.498956 9330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:10:59.499241 9330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:10:59.505119 9338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:10:59.505119 9335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:59.505357 9330 server_base.cc:1034] running on GCE node
W20260811 10:10:59.505355 9336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:10:59.505703 9330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:10:59.505954 9330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:10:59.507135 9330 hybrid_clock.cc:648] HybridClock initialized: now 1786443059507107 us; error 44 us; skew 500 ppm
Aug 11 10:10:59 dist-test-slave-g350 krb5kdc[7682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443059, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:10:59.510005 9330 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:10:59.510617 9330 webserver.cc:460] Webserver started at http://127.22.177.3:38361/ using document root <none> and password file <none>
I20260811 10:10:59.510950 9330 fs_manager.cc:359] Metadata directory not provided
I20260811 10:10:59.511026 9330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:10:59.512648 9330 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:59.513518 9345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.513880 9330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:10:59.514050 9330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "1bd45df631ba4211b9d9d902c53beefd"
format_stamp: "Formatted at 2026-08-11 10:10:17 on dist-test-slave-g350"
server_key: "dbf46e03a9d30820bd92e8581192a414"
server_key_iv: "88029766e602d82568d361c693e1af19"
server_key_version: "encryptionkey@0"
I20260811 10:10:59.514556 9330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:10:59.529577 9330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:10:59.532407 9330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:10:59.532596 9330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:10:59.532883 9330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:10:59.533231 9330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:10:59.533290 9330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.533342 9330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:10:59.533381 9330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:10:59.541030 9330 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:45295
I20260811 10:10:59.541273 9477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:45295 every 8 connection(s)
I20260811 10:10:59.541551 9330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:10:59.542173 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 9330
I20260811 10:10:59.542986 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.542363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:49879)
0811 10:10:59.542656 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.542743 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
W20260811 10:10:59.543448 9478 heartbeater.cc:644] Failed to heartbeat to 127.22.177.62:36473 (0 consecutive failures): Network error: Failed to ping master at 127.22.177.62:36473: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
I20260811 10:10:59.543910 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.543793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:51949)
0811 10:10:59.543833 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.543886 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 10:10:59.544273 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:10:59.544170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:57301)
0811 10:10:59.544230 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 10:10:59.544254 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 10:10:59.544420 9478 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:11:00.338280 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:00.338021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:38955)
0811 10:11:00.338175 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:00.338214 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260811 10:11:00.404763 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:00.404445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:54753)
0811 10:11:00.404612 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:00.404658 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260811 10:11:00.545136 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:00.544869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:38695)
0811 10:11:00.545030 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:00.545074 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260811 10:11:01.032656 9485 raft_consensus.cc:482] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:01.032815 9485 raft_consensus.cc:504] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:11:01.033305 9485 leader_election.cc:288] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295), 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823)
Aug 11 10:11:01 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443059, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.2@KRBTEST.COM
Aug 11 10:11:01 dist-test-slave-g350 krb5kdc[7682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443059, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.3@KRBTEST.COM
I20260811 10:11:01.047637 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.033691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:33205 (local address 127.22.177.3:45295)
0811 10:11:01.033881 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:01.033894 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:01.033960 (+ 66us) server_negotiation.cc:407] Connection header received
0811 10:11:01.034670 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:01.034673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:01.034717 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:01.034833 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:01.035942 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.037567 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.039219 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.039861 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.039990 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:01.043326 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:01.043329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:01.043330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:01.043342 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:01.045638 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:01.046273 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:01.046277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:01.046277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:01.046387 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:01.046682 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:01.046683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:01.046684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:01.046946 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:01.047039 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:01.047361 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 10:11:01.047512 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":57,"threads_started":1}
I20260811 10:11:01.047636 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.033699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:39787 (local address 127.22.177.2:35823)
0811 10:11:01.033881 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:01.033894 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:01.033924 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:11:01.034566 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:01.034570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:01.034632 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:01.034810 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:01.035942 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.037520 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.039222 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.039904 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.040003 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:01.043316 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:01.043321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:01.043321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:01.043333 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:01.045668 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:01.046273 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:01.046277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:01.046277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:01.046390 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:01.046675 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:01.046676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:01.046676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:01.046946 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:01.047053 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:01.047354 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 10:11:01.047514 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260811 10:11:01.047626 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.033528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45295 (local address 127.22.177.1:33205)
0811 10:11:01.033889 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:01.033901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:01.033972 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:01.034918 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:01.034944 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:01.035488 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:01.035786 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:01.035795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.037734 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.037735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:01.039096 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:01.039099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.040004 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.040005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:01.040166 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:01.040864 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:01.040875 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:01.043197 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:01.045809 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:01.045811 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:01.045811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:01.046163 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:01.046462 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:01.046463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:01.046464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:01.046600 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:01.047023 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:01.047031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:01.047178 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 10:11:01.047298 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 10:11:01.047528 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":122,"thread_start_us":92,"threads_started":1}
I20260811 10:11:01.047770 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.033520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35823 (local address 127.22.177.1:39787)
0811 10:11:01.033676 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:01.033689 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:01.033779 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:01.034918 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:01.034921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:01.035496 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:01.035786 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:01.035795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.037735 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.037735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:01.039095 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:01.039099 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:01.040002 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:01.040003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:01.040213 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:01.040864 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:01.040875 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:01.042871 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:01.045774 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:01.045784 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:01.045787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:01.046160 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:01.046485 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:01.046486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:01.046486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:01.046601 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:01.047050 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:01.047051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:01.047178 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 10:11:01.047297 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 10:11:01.047613 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":215,"thread_start_us":89,"threads_started":1}
W20260811 10:11:01.048815 9043 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
W20260811 10:11:01.048945 9040 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
I20260811 10:11:01.048988 9040 leader_election.cc:302] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2389cdd77e2344a189a8b6e456aeee3e; no voters: 1bd45df631ba4211b9d9d902c53beefd, 81663a104e7248f18ef16bcb25ebf08e
I20260811 10:11:01.049106 9485 raft_consensus.cc:2726] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:11:01.339092 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.338897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:55695)
0811 10:11:01.338985 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:01.339037 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 10:11:01.405673 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.405348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:57253)
0811 10:11:01.405499 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:01.405559 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260811 10:11:01.546201 9492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:01.545904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:49305)
0811 10:11:01.546083 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:01.546133 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260811 10:11:02.339892 9494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:02.339654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:49617)
0811 10:11:02.339806 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:02.339844 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260811 10:11:02.406424 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:02.406200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:54201)
0811 10:11:02.406337 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:02.406378 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260811 10:11:02.547252 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:02.546896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:54647)
0811 10:11:02.547152 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:02.547200 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":80,"threads_started":1}
I20260811 10:11:03.005537 9499 raft_consensus.cc:482] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:03.005734 9499 raft_consensus.cc:504] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:11:03.005978 9499 leader_election.cc:288] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295), 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823)
W20260811 10:11:03.006472 9043 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
W20260811 10:11:03.006564 9040 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
I20260811 10:11:03.006667 9040 leader_election.cc:302] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2389cdd77e2344a189a8b6e456aeee3e; no voters: 1bd45df631ba4211b9d9d902c53beefd, 81663a104e7248f18ef16bcb25ebf08e
I20260811 10:11:03.006774 9499 raft_consensus.cc:2726] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:11:03.340610 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:03.340395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:34065)
0811 10:11:03.340526 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:03.340562 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260811 10:11:03.407294 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:03.406978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:57945)
0811 10:11:03.407108 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:03.407156 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260811 10:11:03.548465 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:03.548119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:44181)
0811 10:11:03.548351 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:03.548412 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":177,"threads_started":1}
I20260811 10:11:04.341514 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:04.341260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:54037)
0811 10:11:04.341420 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:04.341467 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
I20260811 10:11:04.408130 9505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:04.407897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:37197)
0811 10:11:04.408042 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:04.408081 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260811 10:11:04.549536 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:04.549154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:42413)
0811 10:11:04.549441 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:04.549485 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":218,"threads_started":1}
I20260811 10:11:04.991891 9508 raft_consensus.cc:482] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:04.992009 9508 raft_consensus.cc:504] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:11:04.992278 9508 leader_election.cc:288] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295), 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823)
W20260811 10:11:04.992861 9043 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
W20260811 10:11:04.992945 9040 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
I20260811 10:11:04.993036 9040 leader_election.cc:302] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2389cdd77e2344a189a8b6e456aeee3e; no voters: 1bd45df631ba4211b9d9d902c53beefd, 81663a104e7248f18ef16bcb25ebf08e
I20260811 10:11:04.993124 9508 raft_consensus.cc:2726] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:11:05.342406 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:05.342086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:43033)
0811 10:11:05.342312 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:05.342355 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":159,"threads_started":1}
I20260811 10:11:05.409008 9511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:05.408730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:55315)
0811 10:11:05.408898 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:05.408958 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260811 10:11:05.550489 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:05.550191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:46203)
0811 10:11:05.550389 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:05.550438 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260811 10:11:06.343201 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:06.342951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:54441)
0811 10:11:06.343108 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:06.343151 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260811 10:11:06.409955 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:06.409702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:33865)
0811 10:11:06.409860 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:06.409906 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260811 10:11:06.551386 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:06.551098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:33623)
0811 10:11:06.551288 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:06.551335 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260811 10:11:07.344058 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:07.343794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:40913)
0811 10:11:07.343958 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:07.344007 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260811 10:11:07.410907 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:07.410617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:41869)
0811 10:11:07.410783 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:07.410833 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260811 10:11:07.552412 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:07.552133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:53375)
0811 10:11:07.552316 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:07.552360 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260811 10:11:08.345007 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:08.344732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:47509)
0811 10:11:08.344907 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:08.344959 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
I20260811 10:11:08.411723 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:08.411442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:49701)
0811 10:11:08.411623 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:08.411667 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260811 10:11:08.553404 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:08.553066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:58171)
0811 10:11:08.553297 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:08.553347 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":105,"threads_started":1}
I20260811 10:11:09.043789 9522 raft_consensus.cc:482] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:09.043907 9522 raft_consensus.cc:504] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd45df631ba4211b9d9d902c53beefd" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45295 } } peers { permanent_uuid: "81663a104e7248f18ef16bcb25ebf08e" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35823 } } peers { permanent_uuid: "2389cdd77e2344a189a8b6e456aeee3e" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 36919 } }
I20260811 10:11:09.044199 9522 leader_election.cc:288] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295), 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823)
W20260811 10:11:09.044552 9043 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1bd45df631ba4211b9d9d902c53beefd (127.22.177.3:45295): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
W20260811 10:11:09.044749 9040 leader_election.cc:341] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81663a104e7248f18ef16bcb25ebf08e (127.22.177.2:35823): Not found: Tablet not found: c643a4fa88a441668d16dd58d97643b3
I20260811 10:11:09.044792 9040 leader_election.cc:302] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2389cdd77e2344a189a8b6e456aeee3e; no voters: 1bd45df631ba4211b9d9d902c53beefd, 81663a104e7248f18ef16bcb25ebf08e
I20260811 10:11:09.044870 9522 raft_consensus.cc:2726] T c643a4fa88a441668d16dd58d97643b3 P 2389cdd77e2344a189a8b6e456aeee3e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:11:09.345867 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:09.345577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:47545)
0811 10:11:09.345765 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:09.345813 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":116,"threads_started":1}
I20260811 10:11:09.412550 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:09.412277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:58115)
0811 10:11:09.412451 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:09.412499 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260811 10:11:09.554330 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:09.554017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:46963)
0811 10:11:09.554225 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:09.554278 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
I20260811 10:11:10.346720 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:10.346469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:37511)
0811 10:11:10.346618 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:10.346661 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260811 10:11:10.413390 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:10.413127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:45439)
0811 10:11:10.413297 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:10.413341 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260811 10:11:10.555287 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:10.555044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:37515)
0811 10:11:10.555186 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:10.555234 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260811 10:11:11.347672 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:11.347362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:43509)
0811 10:11:11.347576 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:11.347613 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":136,"threads_started":1}
I20260811 10:11:11.414157 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:11.413883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:41299)
0811 10:11:11.414062 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:11.414108 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260811 10:11:11.556203 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:11.555937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:37311)
0811 10:11:11.556099 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:11.556154 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260811 10:11:12.348616 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:12.348348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:49509)
0811 10:11:12.348514 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:12.348566 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260811 10:11:12.414925 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:12.414652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:59409)
0811 10:11:12.414843 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:12.414877 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
I20260811 10:11:12.557094 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:12.556824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:40135)
0811 10:11:12.556990 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:12.557036 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":69,"threads_started":1}
I20260811 10:11:13.349541 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:13.349232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:37459)
0811 10:11:13.349439 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:13.349487 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
I20260811 10:11:13.415688 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:13.415453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:37381)
0811 10:11:13.415607 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:13.415641 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260811 10:11:13.558220 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:13.557932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:60271)
0811 10:11:13.558119 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:13.558167 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":116,"threads_started":1}
I20260811 10:11:14.350755 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:14.350499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.1:42063)
0811 10:11:14.350657 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:14.350707 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260811 10:11:14.416530 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:14.416307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.2:54609)
0811 10:11:14.416449 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:14.416484 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260811 10:11:14.552417 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 8983
I20260811 10:11:14.559196 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:14.558965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:36473 (local address 127.22.177.3:34779)
0811 10:11:14.559103 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:14.559147 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.62:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260811 10:11:14.561520 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 9173
I20260811 10:11:14.571606 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 9330
2026-08-11T10:11:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58691 ms)
[----------] 30 tests from SecurityITest (248968 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:14 dist-test-slave-g350 krb5kdc[9544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:14 dist-test-slave-g350 krb5kdc[9544](info): set up 2 sockets
Aug 11 10:11:14 dist-test-slave-g350 krb5kdc[9544](info): commencing operation
krb5kdc: starting...
W20260811 10:11:15.769768 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:15 dist-test-slave-g350 krb5kdc[9544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:15.913357 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:39653
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:39653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:16.004566 9560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:16.004834 9560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:16.004875 9560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:16.006106 9560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:16.006181 9560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:16.006206 9560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:16.006227 9560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:16.006256 9560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:16.007946 9560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:39653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:39653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:16.008244 9560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:16.008488 9560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:16.013916 9565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.014027 9566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.013916 9569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:16.014056 9560 server_base.cc:1034] running on GCE node
I20260811 10:11:16.014542 9560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:16.014856 9560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:16.016018 9560 hybrid_clock.cc:648] HybridClock initialized: now 1786443076016005 us; error 29 us; skew 500 ppm
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:16.018844 9560 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:16.019306 9560 webserver.cc:460] Webserver started at http://127.22.177.62:34915/ using document root <none> and password file <none>
I20260811 10:11:16.019584 9560 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:16.019658 9560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:16.019811 9560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:16.020978 9560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "7363adba03354b928b6af3caa6917d08"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "b638441c04a50396910c9eee84428061"
server_key_iv: "a7a88da1513bb842a133ff2844014919"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.021363 9560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "7363adba03354b928b6af3caa6917d08"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "b638441c04a50396910c9eee84428061"
server_key_iv: "a7a88da1513bb842a133ff2844014919"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.022821 9560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:16.023624 9578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.023790 9560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:11:16.023906 9560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "7363adba03354b928b6af3caa6917d08"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "b638441c04a50396910c9eee84428061"
server_key_iv: "a7a88da1513bb842a133ff2844014919"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.023983 9560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:16.042090 9560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:16.044065 9560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:16.044179 9560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:16.048245 9560 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:39653
I20260811 10:11:16.048321 9642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:39653 every 8 connection(s)
I20260811 10:11:16.048687 9560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:16.049701 9643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:16.050092 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 9560
I20260811 10:11:16.050209 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:16.050423 23236 external_mini_cluster.cc:1428] Setting key 9c126e362e8f29bcbb26b4c4ae68aa4b
I20260811 10:11:16.052860 9643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08: Bootstrap starting.
I20260811 10:11:16.053486 9643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:16.053900 9643 log.cc:824] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:16.054730 9643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08: No bootstrap required, opened a new log
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:16.057437 9643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7363adba03354b928b6af3caa6917d08" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39653 } }
I20260811 10:11:16.057686 9643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:16.057775 9643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7363adba03354b928b6af3caa6917d08, State: Initialized, Role: FOLLOWER
I20260811 10:11:16.057924 9643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7363adba03354b928b6af3caa6917d08" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39653 } }
I20260811 10:11:16.058029 9643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:16.058089 9643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:16.058139 9643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:16.058882 9643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7363adba03354b928b6af3caa6917d08" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39653 } }
I20260811 10:11:16.058974 9643 leader_election.cc:302] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7363adba03354b928b6af3caa6917d08; no voters:
I20260811 10:11:16.059181 9643 leader_election.cc:288] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:16.059477 9643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:16.059597 9649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:16.059810 9649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [term 1 LEADER]: Becoming Leader. State: Replica: 7363adba03354b928b6af3caa6917d08, State: Running, Role: LEADER
I20260811 10:11:16.059960 9649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7363adba03354b928b6af3caa6917d08" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39653 } }
I20260811 10:11:16.060390 9649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7363adba03354b928b6af3caa6917d08. Latest consensus state: current_term: 1 leader_uuid: "7363adba03354b928b6af3caa6917d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7363adba03354b928b6af3caa6917d08" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39653 } } }
I20260811 10:11:16.060396 9650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7363adba03354b928b6af3caa6917d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7363adba03354b928b6af3caa6917d08" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 39653 } } }
I20260811 10:11:16.060520 9649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:16.060539 9650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:16.060789 9662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:16.061519 9662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:16.062650 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.051004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.22.177.62:39653)
0811 10:11:16.051205 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.051213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.051242 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:11:16.051682 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.051689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.051762 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.051878 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:16.052703 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.053675 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.054675 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.055139 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.055224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.058122 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:16.058129 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:16.058129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:16.058144 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:16.060652 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.061143 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.061147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.061147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.061269 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.061533 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.061534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.061535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.061941 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:16.062117 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.062261 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.062430 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260811 10:11:16.063524 9662 catalog_manager.cc:1347] Generated new cluster ID: 6462eb3736f54b24bd5b38ec09f4e195
I20260811 10:11:16.063585 9662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:16.083902 9662 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:16.084793 9662 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:16.087786 9662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08: Generated new TSK 0
I20260811 10:11:16.088003 9662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:11:16.122617 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:39653
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:16.211223 9671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:16.211433 9671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:16.211458 9671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:16.212531 9671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:16.212565 9671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:16.212590 9671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:11:16.214031 9671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:39653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:16.214228 9671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:16.214413 9671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:11:16.219903 9671 server_base.cc:1034] running on GCE node
W20260811 10:11:16.219811 9677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.219918 9678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.219839 9680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:16.220220 9671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:16.220507 9671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:16.221658 9671 hybrid_clock.cc:648] HybridClock initialized: now 1786443076221653 us; error 43 us; skew 500 ppm
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:16.224191 9671 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:11:16.224659 9671 webserver.cc:460] Webserver started at http://127.22.177.1:39511/ using document root <none> and password file <none>
I20260811 10:11:16.224920 9671 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:16.224967 9671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:16.225101 9671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:16.226306 9671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "cf5021b4be584b9d98c776444fec7b47"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "3159b8b623686502c69d87eda14d1ec9"
server_key_iv: "e6c1ebe93666f543c6483b754414e0fe"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.226687 9671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "cf5021b4be584b9d98c776444fec7b47"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "3159b8b623686502c69d87eda14d1ec9"
server_key_iv: "e6c1ebe93666f543c6483b754414e0fe"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.228188 9671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:16.229066 9687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.229365 9671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:16.229439 9671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "cf5021b4be584b9d98c776444fec7b47"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "3159b8b623686502c69d87eda14d1ec9"
server_key_iv: "e6c1ebe93666f543c6483b754414e0fe"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.229530 9671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:16.249368 9671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:16.251976 9671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:16.252228 9671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:16.252539 9671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:16.253001 9671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:16.253067 9671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.253109 9671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:16.253135 9671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.259667 9671 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:43171
I20260811 10:11:16.259796 9818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:43171 every 8 connection(s)
I20260811 10:11:16.260154 9671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:16.269207 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 9671
I20260811 10:11:16.269342 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:11:16.269632 23236 external_mini_cluster.cc:1428] Setting key 1b73929c09424f28ecb7adc78b6734e3
I20260811 10:11:16.271411 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.260970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:48093 (local address 127.22.177.62:39653)
0811 10:11:16.261061 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.261064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.261383 (+ 319us) server_negotiation.cc:407] Connection header received
0811 10:11:16.261921 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.261923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.261934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.261991 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:16.262981 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.263489 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.264679 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.264902 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.264963 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.267643 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:16.267645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:16.267645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:16.267652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:16.269488 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.270131 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.270132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.270133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.270207 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.270640 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.270641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.270641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.270795 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:16.270894 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.271208 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.271319 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 10:11:16.271628 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.260935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:39653 (local address 127.22.177.1:48093)
0811 10:11:16.261129 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.261274 (+ 145us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.261811 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.262048 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.262051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.262675 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:16.262891 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.262896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.263601 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.263602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.264581 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.264584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.264992 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.264993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.265112 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.265875 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:16.265884 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:16.267539 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:16.269582 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:16.269611 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:16.269633 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:16.270041 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:16.270314 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:16.270315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:16.270315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:16.270499 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:16.270888 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:16.270898 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:16.271057 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.271173 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.271366 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260811 10:11:16.272400 9819 heartbeater.cc:347] Connected to a master server at 127.22.177.62:39653
I20260811 10:11:16.272531 9819 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:16.272773 9819 heartbeater.cc:511] Master 127.22.177.62:39653 requested a full tablet report, sending...
I20260811 10:11:16.273451 9600 ts_manager.cc:194] Registered new tserver with Master: cf5021b4be584b9d98c776444fec7b47 (127.22.177.1:43171)
I20260811 10:11:16.274170 9600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:48093
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:11:16.301986 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:39653
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:16.388468 9824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:16.388675 9824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:16.388700 9824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:16.389815 9824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:16.389850 9824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:16.389876 9824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:11:16.391268 9824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:39653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:16.391471 9824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:16.391644 9824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:16.397131 9833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.397154 9832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.397140 9835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:16.397300 9824 server_base.cc:1034] running on GCE node
I20260811 10:11:16.397697 9824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:16.398008 9824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:16.399235 9824 hybrid_clock.cc:648] HybridClock initialized: now 1786443076399217 us; error 31 us; skew 500 ppm
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:16.402057 9824 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:11:16.402562 9824 webserver.cc:460] Webserver started at http://127.22.177.2:40717/ using document root <none> and password file <none>
I20260811 10:11:16.402761 9824 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:16.402812 9824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:16.402904 9824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:16.403972 9824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "7ab6435554454a80a47dfab6cf4d44fd"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "b21dcb4b927dfd1d1e0d864a8c55ee1f"
server_key_iv: "5ab801b444e3d411e04f0fa207522298"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.404289 9824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "7ab6435554454a80a47dfab6cf4d44fd"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "b21dcb4b927dfd1d1e0d864a8c55ee1f"
server_key_iv: "5ab801b444e3d411e04f0fa207522298"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.405587 9824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:16.406481 9844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.406739 9824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:16.406865 9824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "7ab6435554454a80a47dfab6cf4d44fd"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "b21dcb4b927dfd1d1e0d864a8c55ee1f"
server_key_iv: "5ab801b444e3d411e04f0fa207522298"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.406944 9824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:16.424099 9824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:16.426353 9824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:16.426515 9824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:16.426762 9824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:16.427146 9824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:16.427183 9824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.427207 9824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:16.427255 9824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.434257 9824 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:37659
I20260811 10:11:16.434346 9973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:37659 every 8 connection(s)
I20260811 10:11:16.434653 9824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:11:16.438717 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 9824
I20260811 10:11:16.438890 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:11:16.439179 23236 external_mini_cluster.cc:1428] Setting key 9837e161b857d7373427ac60a67fc435
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:16.445960 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.435493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:33533 (local address 127.22.177.62:39653)
0811 10:11:16.435555 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.435558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.435746 (+ 188us) server_negotiation.cc:407] Connection header received
0811 10:11:16.436314 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.436315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.436325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.436394 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:16.437507 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.438234 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.439755 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.440039 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.440122 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.442667 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:16.442669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:16.442669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:16.442677 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:16.444177 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.444781 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.444783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.444783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.444843 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.445189 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.445189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.445190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.445346 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:16.445399 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.445821 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.445892 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:11:16.446382 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.435440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:39653 (local address 127.22.177.2:33533)
0811 10:11:16.435610 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.435655 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.436219 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.436448 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.436452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.437137 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:16.437393 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.437399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.438335 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.438336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.439658 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.439661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.440131 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.440132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.440281 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.440879 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:16.440888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:16.442550 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:16.444284 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:16.444293 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:16.444296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:16.444692 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:16.444923 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:16.444925 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:16.444925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:16.445113 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:16.445422 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:16.445430 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:16.445657 (+ 227us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.445808 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.446026 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260811 10:11:16.447070 9974 heartbeater.cc:347] Connected to a master server at 127.22.177.62:39653
I20260811 10:11:16.447175 9974 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:16.447381 9974 heartbeater.cc:511] Master 127.22.177.62:39653 requested a full tablet report, sending...
I20260811 10:11:16.447891 9600 ts_manager.cc:194] Registered new tserver with Master: 7ab6435554454a80a47dfab6cf4d44fd (127.22.177.2:37659)
I20260811 10:11:16.448287 9600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:33533
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:11:16.474562 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:39653
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:38225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:16.568552 9980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:16.568806 9980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:16.568837 9980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:16.570359 9980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:16.570420 9980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:16.570482 9980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:11:16.572142 9980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:38225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:39653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:16.572376 9980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:16.572595 9980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:16.578631 9988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:16.578747 9980 server_base.cc:1034] running on GCE node
W20260811 10:11:16.578625 9986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.578625 9985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:16.579101 9980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:16.579375 9980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:16.580554 9980 hybrid_clock.cc:648] HybridClock initialized: now 1786443076580517 us; error 59 us; skew 500 ppm
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:16.583518 9980 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:11:16.583971 9980 webserver.cc:460] Webserver started at http://127.22.177.3:42295/ using document root <none> and password file <none>
I20260811 10:11:16.584208 9980 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:16.584257 9980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:16.584357 9980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:16.585685 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "6457704212a4443b8bc9afcb62963c09"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "8f6b2592214d869822f7d43bb46f969d"
server_key_iv: "f400a64366f29fedebc90a71c4778d6d"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.586055 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "6457704212a4443b8bc9afcb62963c09"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "8f6b2592214d869822f7d43bb46f969d"
server_key_iv: "f400a64366f29fedebc90a71c4778d6d"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.587548 9980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:11:16.588578 9998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.588791 9980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:16.588882 9980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "6457704212a4443b8bc9afcb62963c09"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "8f6b2592214d869822f7d43bb46f969d"
server_key_iv: "f400a64366f29fedebc90a71c4778d6d"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.588939 9980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:16.606416 9980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:16.608659 9980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:16.608798 9980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:16.609002 9980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:16.609378 9980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:16.609417 9980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.609440 9980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:16.609452 9980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.616120 9980 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:45113
I20260811 10:11:16.616218 10125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:45113 every 8 connection(s)
I20260811 10:11:16.616631 9980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:11:16.621449 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 9980
I20260811 10:11:16.621632 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:11:16.621912 23236 external_mini_cluster.cc:1428] Setting key a5410fb80b67acb208ddfe119e45bcb7
Aug 11 10:11:16 dist-test-slave-g350 krb5kdc[9544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:16.626708 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.617452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:35763 (local address 127.22.177.62:39653)
0811 10:11:16.617497 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.617499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.617773 (+ 274us) server_negotiation.cc:407] Connection header received
0811 10:11:16.618237 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.618237 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.618247 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.618310 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:16.619326 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.619875 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.621099 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.621321 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.621423 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.624056 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:16.624057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:16.624057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:16.624065 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:16.625363 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.625877 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.625878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.625879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.625933 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.626151 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.626151 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.626151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.626251 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:16.626299 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.626568 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.626643 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:16.626911 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.617407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:39653 (local address 127.22.177.3:35763)
0811 10:11:16.617586 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.617662 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.618155 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.618359 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.618362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.618954 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:16.619227 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.619233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.619990 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.619991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.620995 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.620998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.621434 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.621435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.621564 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.622321 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:16.622329 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:16.623956 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:16.625462 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:16.625469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:16.625471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:16.625804 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:16.626000 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:16.626000 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:16.626001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:16.626107 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:16.626301 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:16.626305 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:16.626425 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.626501 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.626684 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260811 10:11:16.627564 10126 heartbeater.cc:347] Connected to a master server at 127.22.177.62:39653
I20260811 10:11:16.627651 10126 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:16.627848 10126 heartbeater.cc:511] Master 127.22.177.62:39653 requested a full tablet report, sending...
I20260811 10:11:16.628374 9600 ts_manager.cc:194] Registered new tserver with Master: 6457704212a4443b8bc9afcb62963c09 (127.22.177.3:45113)
I20260811 10:11:16.628758 9600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:35763
I20260811 10:11:16.633971 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:11:16.642217 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.635419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.22.177.62:39653)
0811 10:11:16.635463 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.635465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.635615 (+ 150us) server_negotiation.cc:407] Connection header received
0811 10:11:16.635666 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.635667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.635675 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.635728 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:16.636464 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.636958 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.637842 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.638026 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.638083 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.639488 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:16.639489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:16.639489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:16.639496 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:16.640948 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.641369 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.641370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.641370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.641425 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:16.641685 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:16.641685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:16.641686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:16.641829 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:16.641908 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.642084 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.642149 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:11:16.644264 9600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:11:16.645923 9600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:11:16.652976 10139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.648731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:45113 (local address 127.0.0.1:34762)
0811 10:11:16.648910 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.648953 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.649029 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.649502 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.649505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.649508 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:16.649585 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.649613 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.650568 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.650570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.651605 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.651608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.652599 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.652600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.652700 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.652718 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.652818 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.652887 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260811 10:11:16.652997 10141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.648918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.22.177.3:45113)
0811 10:11:16.649148 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.649153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.649165 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:16.649290 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.649292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.649311 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.649398 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:16.649716 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.650449 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.651718 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.652483 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.652555 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.652619 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.652856 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.652909 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
I20260811 10:11:16.653712 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.648743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43171 (local address 127.0.0.1:54934)
0811 10:11:16.649101 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.649114 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.649185 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.649678 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.649679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.649682 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:16.649747 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.649748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.651271 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.651272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.652350 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.652354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.653429 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.653430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.653502 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.653507 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.653558 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.653627 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":99,"threads_started":1}
I20260811 10:11:16.653962 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.648879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.22.177.1:43171)
0811 10:11:16.649418 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.649422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.649432 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:16.649478 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.649480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.649497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.649850 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:16.649946 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.651150 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.652465 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.653335 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.653412 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.653466 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.653835 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.653883 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":102,"threads_started":1}
I20260811 10:11:16.654304 10051 tablet_service.cc:1467] Processing CreateTablet for tablet a8d631249838415f9a5bbded2183447c (DEFAULT_TABLE table=test-table [id=a77223b4e0484f83a38a1f0e4842d557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:16.654690 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.649223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.22.177.2:37659)
0811 10:11:16.649416 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.649421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.649432 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:11:16.649481 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.649483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.649498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.649585 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:16.649841 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.651121 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.653590 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.654334 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.654399 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.654436 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.654591 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.654626 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260811 10:11:16.654697 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.648971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:37659 (local address 127.0.0.1:48776)
0811 10:11:16.649206 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.649216 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.649264 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.649699 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.649700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.649702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:16.649745 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.649747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.652460 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.652461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.653478 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.653481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.654428 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.654429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.654492 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.654497 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.654564 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.654606 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":73,"threads_started":1}
I20260811 10:11:16.654748 10051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d631249838415f9a5bbded2183447c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:16.655364 9897 tablet_service.cc:1467] Processing CreateTablet for tablet a8d631249838415f9a5bbded2183447c (DEFAULT_TABLE table=test-table [id=a77223b4e0484f83a38a1f0e4842d557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:16.655609 9897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d631249838415f9a5bbded2183447c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:16.657688 10144 tablet_bootstrap.cc:492] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Bootstrap starting.
I20260811 10:11:16.658407 10144 tablet_bootstrap.cc:654] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:16.658394 9746 tablet_service.cc:1467] Processing CreateTablet for tablet a8d631249838415f9a5bbded2183447c (DEFAULT_TABLE table=test-table [id=a77223b4e0484f83a38a1f0e4842d557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:16.658614 10145 tablet_bootstrap.cc:492] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Bootstrap starting.
I20260811 10:11:16.658795 9746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d631249838415f9a5bbded2183447c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:16.659063 10144 log.cc:824] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:16.659243 10145 tablet_bootstrap.cc:654] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:16.659876 10145 log.cc:824] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:16.660005 10144 tablet_bootstrap.cc:492] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: No bootstrap required, opened a new log
I20260811 10:11:16.660109 10144 ts_tablet_manager.cc:1397] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:11:16.662328 10144 raft_consensus.cc:348] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.662510 10148 tablet_bootstrap.cc:492] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47: Bootstrap starting.
I20260811 10:11:16.662544 10144 raft_consensus.cc:374] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:16.662587 10144 raft_consensus.cc:729] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6457704212a4443b8bc9afcb62963c09, State: Initialized, Role: FOLLOWER
I20260811 10:11:16.662715 10144 consensus_queue.cc:260] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.663043 10144 ts_tablet_manager.cc:1428] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:11:16.663110 10126 heartbeater.cc:503] Master 127.22.177.62:39653 was elected leader, sending a full tablet report...
I20260811 10:11:16.663381 10148 tablet_bootstrap.cc:654] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:16.663724 10145 tablet_bootstrap.cc:492] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: No bootstrap required, opened a new log
I20260811 10:11:16.663868 10145 ts_tablet_manager.cc:1397] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 10:11:16.663972 10148 log.cc:824] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:16.664824 10148 tablet_bootstrap.cc:492] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47: No bootstrap required, opened a new log
I20260811 10:11:16.664902 10148 ts_tablet_manager.cc:1397] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:11:16.666147 10145 raft_consensus.cc:348] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.666365 10145 raft_consensus.cc:374] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:16.666404 10145 raft_consensus.cc:729] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab6435554454a80a47dfab6cf4d44fd, State: Initialized, Role: FOLLOWER
I20260811 10:11:16.666527 10145 consensus_queue.cc:260] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.666838 10145 ts_tablet_manager.cc:1428] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:16.666864 9974 heartbeater.cc:503] Master 127.22.177.62:39653 was elected leader, sending a full tablet report...
I20260811 10:11:16.667034 10148 raft_consensus.cc:348] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.667161 10148 raft_consensus.cc:374] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:16.667223 10148 raft_consensus.cc:729] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf5021b4be584b9d98c776444fec7b47, State: Initialized, Role: FOLLOWER
I20260811 10:11:16.667340 10148 consensus_queue.cc:260] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.667624 10148 ts_tablet_manager.cc:1428] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:16.667629 9819 heartbeater.cc:503] Master 127.22.177.62:39653 was elected leader, sending a full tablet report...
W20260811 10:11:16.685352 9976 tablet.cc:2369] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:11:16.696003 10149 raft_consensus.cc:482] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:16.696179 10149 raft_consensus.cc:504] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.696540 10149 leader_election.cc:288] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf5021b4be584b9d98c776444fec7b47 (127.22.177.1:43171), 7ab6435554454a80a47dfab6cf4d44fd (127.22.177.2:37659)
I20260811 10:11:16.701269 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.696891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:57097 (local address 127.22.177.1:43171)
0811 10:11:16.696973 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.696976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.696988 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:16.697029 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.697030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.697047 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.697147 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:16.697400 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.698342 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.700134 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.700767 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.700865 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.700878 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.701135 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.701183 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 10:11:16.701296 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.696882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:37659 (local address 127.22.177.3:36115)
0811 10:11:16.697065 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.697074 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.697151 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.697394 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.697395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.697397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:16.697463 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.697464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.698471 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.698472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.699979 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.699983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.700925 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.700926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.701024 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.701031 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.701108 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.701169 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":75,"thread_start_us":127,"threads_started":1}
I20260811 10:11:16.701330 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.697096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:36115 (local address 127.22.177.2:37659)
0811 10:11:16.697231 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.697233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.697243 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:16.697281 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.697281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.697293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.697389 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:16.697547 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.698362 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.700127 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.700817 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.700888 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.700897 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.701134 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.701252 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 10:11:16.701445 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.696753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43171 (local address 127.22.177.3:57097)
0811 10:11:16.696815 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.696828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:16.696875 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:16.697163 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:16.697164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:16.697166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:16.697296 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.697297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.698471 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.698472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.699979 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:16.699983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.700924 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.700926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:16.701031 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.701034 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:16.701108 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:11:16.701169 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":122}
I20260811 10:11:16.701766 9920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d631249838415f9a5bbded2183447c" candidate_uuid: "6457704212a4443b8bc9afcb62963c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab6435554454a80a47dfab6cf4d44fd" is_pre_election: true
I20260811 10:11:16.701957 9769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d631249838415f9a5bbded2183447c" candidate_uuid: "6457704212a4443b8bc9afcb62963c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf5021b4be584b9d98c776444fec7b47" is_pre_election: true
I20260811 10:11:16.702003 9920 raft_consensus.cc:2454] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6457704212a4443b8bc9afcb62963c09 in term 0.
I20260811 10:11:16.702090 9769 raft_consensus.cc:2454] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6457704212a4443b8bc9afcb62963c09 in term 0.
I20260811 10:11:16.702337 10003 leader_election.cc:302] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6457704212a4443b8bc9afcb62963c09, 7ab6435554454a80a47dfab6cf4d44fd; no voters:
I20260811 10:11:16.702512 10149 raft_consensus.cc:2781] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:11:16.702600 10149 raft_consensus.cc:482] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:11:16.702643 10149 raft_consensus.cc:3037] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:16.703604 10149 raft_consensus.cc:504] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.703809 10149 leader_election.cc:288] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [CANDIDATE]: Term 1 election: Requested vote from peers cf5021b4be584b9d98c776444fec7b47 (127.22.177.1:43171), 7ab6435554454a80a47dfab6cf4d44fd (127.22.177.2:37659)
I20260811 10:11:16.704031 9769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d631249838415f9a5bbded2183447c" candidate_uuid: "6457704212a4443b8bc9afcb62963c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf5021b4be584b9d98c776444fec7b47"
I20260811 10:11:16.704077 9920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d631249838415f9a5bbded2183447c" candidate_uuid: "6457704212a4443b8bc9afcb62963c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab6435554454a80a47dfab6cf4d44fd"
I20260811 10:11:16.704164 9769 raft_consensus.cc:3037] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:16.704200 9920 raft_consensus.cc:3037] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:16.704957 9769 raft_consensus.cc:2454] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6457704212a4443b8bc9afcb62963c09 in term 1.
I20260811 10:11:16.705214 9920 raft_consensus.cc:2454] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6457704212a4443b8bc9afcb62963c09 in term 1.
I20260811 10:11:16.705314 10003 leader_election.cc:302] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6457704212a4443b8bc9afcb62963c09, cf5021b4be584b9d98c776444fec7b47; no voters:
I20260811 10:11:16.705513 10149 raft_consensus.cc:2781] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:16.705802 10149 raft_consensus.cc:686] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 1 LEADER]: Becoming Leader. State: Replica: 6457704212a4443b8bc9afcb62963c09, State: Running, Role: LEADER
I20260811 10:11:16.705941 10149 consensus_queue.cc:237] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } }
I20260811 10:11:16.707006 9599 catalog_manager.cc:5495] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6457704212a4443b8bc9afcb62963c09 (127.22.177.3). New cstate: current_term: 1 leader_uuid: "6457704212a4443b8bc9afcb62963c09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6457704212a4443b8bc9afcb62963c09" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 45113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 } health_report { overall_health: UNKNOWN } } }
I20260811 10:11:16.728245 10141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.725142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.22.177.3:45113)
0811 10:11:16.725209 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.725211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.725221 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:16.725270 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.725271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.725303 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.725372 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:16.725563 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.726213 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.727141 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.727400 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.727474 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.727726 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:16.727854 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:16.727932 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.728137 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.728167 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:11:16.730773 9769 raft_consensus.cc:1261] T a8d631249838415f9a5bbded2183447c P cf5021b4be584b9d98c776444fec7b47 [term 1 FOLLOWER]: Refusing update from remote peer 6457704212a4443b8bc9afcb62963c09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:16.730816 9920 raft_consensus.cc:1261] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 1 FOLLOWER]: Refusing update from remote peer 6457704212a4443b8bc9afcb62963c09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:16.731356 10156 consensus_queue.cc:1035] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf5021b4be584b9d98c776444fec7b47" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:16.731518 10149 consensus_queue.cc:1035] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ab6435554454a80a47dfab6cf4d44fd" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:16.734977 10161 mvcc.cc:205] Tried to move back new op lower bound from 7317270842285285376 to 7317270842188582912. Current Snapshot: MvccSnapshot[applied={T|T < 7317270842285285376}]
I20260811 10:11:16.737062 9599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:16.737177 9599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:16.738070 9599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7363adba03354b928b6af3caa6917d08: Sending DeleteTablet for 3 replicas of tablet a8d631249838415f9a5bbded2183447c
I20260811 10:11:16.738420 9746 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d631249838415f9a5bbded2183447c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:16 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:54934
I20260811 10:11:16.738433 10051 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d631249838415f9a5bbded2183447c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:16 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:34762
I20260811 10:11:16.738675 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 9671
I20260811 10:11:16.738775 9897 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d631249838415f9a5bbded2183447c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:16 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:48776
I20260811 10:11:16.740518 10170 tablet_replica.cc:330] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: stopping tablet replica
I20260811 10:11:16.740635 10170 raft_consensus.cc:2229] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:11:16.740747 10170 raft_consensus.cc:2258] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:16.741328 10170 ts_tablet_manager.cc:1905] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:16.743503 10170 ts_tablet_manager.cc:1918] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:16.743635 10170 log.cc:1196] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/a8d631249838415f9a5bbded2183447c
I20260811 10:11:16.743942 10170 ts_tablet_manager.cc:1939] T a8d631249838415f9a5bbded2183447c P 6457704212a4443b8bc9afcb62963c09: Deleting consensus metadata
I20260811 10:11:16.744493 9580 catalog_manager.cc:4841] TS 6457704212a4443b8bc9afcb62963c09 (127.22.177.3:45113): tablet a8d631249838415f9a5bbded2183447c (table test-table [id=a77223b4e0484f83a38a1f0e4842d557]) successfully deleted
I20260811 10:11:16.749809 10171 tablet_replica.cc:330] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: stopping tablet replica
I20260811 10:11:16.749976 10171 raft_consensus.cc:2229] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:16.750168 10171 raft_consensus.cc:2258] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:16.750761 10171 ts_tablet_manager.cc:1905] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:16.752744 10171 ts_tablet_manager.cc:1918] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:16.752844 10171 log.cc:1196] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/a8d631249838415f9a5bbded2183447c
I20260811 10:11:16.753172 10171 ts_tablet_manager.cc:1939] T a8d631249838415f9a5bbded2183447c P 7ab6435554454a80a47dfab6cf4d44fd: Deleting consensus metadata
I20260811 10:11:16.753806 9583 catalog_manager.cc:4841] TS 7ab6435554454a80a47dfab6cf4d44fd (127.22.177.2:37659): tablet a8d631249838415f9a5bbded2183447c (table test-table [id=a77223b4e0484f83a38a1f0e4842d557]) successfully deleted
W20260811 10:11:16.755805 9583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:43171 (error 108)
I20260811 10:11:16.756093 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 9824
I20260811 10:11:16.756506 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.756367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43171 (local address 127.0.0.1:54946)
0811 10:11:16.756419 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:16.756463 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 10:11:16.756623 9583 catalog_manager.cc:4584] TS cf5021b4be584b9d98c776444fec7b47 (127.22.177.1:43171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8d631249838415f9a5bbded2183447c: Network error: Client connection negotiation failed: client connection to 127.22.177.1:43171: connect: Connection refused (error 111)
I20260811 10:11:16.767602 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 9980
I20260811 10:11:16.776459 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 9560
2026-08-11T10:11:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2201 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T10:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:16Z Disabled control of system clock
I20260811 10:11:16.810271 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37249
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:16.906737 10176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:16.906991 10176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:16.907027 10176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:16.908222 10176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:16.908290 10176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:16.908315 10176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:16.908347 10176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:16.908380 10176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:16.909883 10176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:16.910161 10176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:16.910501 10176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:16.915517 10182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.915657 10184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:16.915669 10181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:16.916185 10176 server_base.cc:1034] running on GCE node
I20260811 10:11:16.916482 10176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:16.916775 10176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:16.917935 10176 hybrid_clock.cc:648] HybridClock initialized: now 1786443076917913 us; error 40 us; skew 500 ppm
I20260811 10:11:16.919251 10176 webserver.cc:460] Webserver started at http://127.22.177.62:33291/ using document root <none> and password file <none>
I20260811 10:11:16.919528 10176 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:16.919600 10176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:16.919742 10176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:16.920858 10176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "dfe76da64edb4619bffe3581f36417fc"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "cdb66f6ea90d7cecc23ceaa73c291032"
server_key_iv: "f800b3820c1f1752a1f4aa76efec86ca"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.921295 10176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "dfe76da64edb4619bffe3581f36417fc"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "cdb66f6ea90d7cecc23ceaa73c291032"
server_key_iv: "f800b3820c1f1752a1f4aa76efec86ca"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.922873 10176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:16.923673 10192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:16.923899 10176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:16.924010 10176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "dfe76da64edb4619bffe3581f36417fc"
format_stamp: "Formatted at 2026-08-11 10:11:16 on dist-test-slave-g350"
server_key: "cdb66f6ea90d7cecc23ceaa73c291032"
server_key_iv: "f800b3820c1f1752a1f4aa76efec86ca"
server_key_version: "encryptionkey@0"
I20260811 10:11:16.924089 10176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:16.931444 10176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:16.931736 10176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:16.931887 10176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:16.936215 10176 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37249
I20260811 10:11:16.936259 10253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37249 every 8 connection(s)
I20260811 10:11:16.936606 10176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:16.937817 10254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:16.941290 10254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: Bootstrap starting.
I20260811 10:11:16.942056 10254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:16.942401 10254 log.cc:824] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:16.943172 10254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: No bootstrap required, opened a new log
I20260811 10:11:16.945470 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 10176
I20260811 10:11:16.945629 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:16.945840 23236 external_mini_cluster.cc:1428] Setting key e79c4544832756c6e816c08d16033a18
I20260811 10:11:16.945922 10254 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfe76da64edb4619bffe3581f36417fc" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37249 } }
I20260811 10:11:16.946117 10254 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:16.946192 10254 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe76da64edb4619bffe3581f36417fc, State: Initialized, Role: FOLLOWER
I20260811 10:11:16.946401 10254 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfe76da64edb4619bffe3581f36417fc" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37249 } }
I20260811 10:11:16.946497 10254 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:16.946565 10254 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:16.946651 10254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:16.947432 10254 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfe76da64edb4619bffe3581f36417fc" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37249 } }
I20260811 10:11:16.947551 10254 leader_election.cc:302] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfe76da64edb4619bffe3581f36417fc; no voters:
I20260811 10:11:16.947728 10254 leader_election.cc:288] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:16.947795 10259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:16.947983 10259 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [term 1 LEADER]: Becoming Leader. State: Replica: dfe76da64edb4619bffe3581f36417fc, State: Running, Role: LEADER
I20260811 10:11:16.948055 10254 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:16.948117 10259 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfe76da64edb4619bffe3581f36417fc" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37249 } }
I20260811 10:11:16.948549 10259 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfe76da64edb4619bffe3581f36417fc. Latest consensus state: current_term: 1 leader_uuid: "dfe76da64edb4619bffe3581f36417fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfe76da64edb4619bffe3581f36417fc" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37249 } } }
I20260811 10:11:16.948757 10259 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:16.948704 10260 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfe76da64edb4619bffe3581f36417fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfe76da64edb4619bffe3581f36417fc" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37249 } } }
I20260811 10:11:16.948980 10260 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc [sys.catalog]: This master's current role is: LEADER
W20260811 10:11:16.950029 10276 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:16.950109 10276 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:16.950186 10268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:16.950418 10268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:16.952047 10268 catalog_manager.cc:1347] Generated new cluster ID: f02134d5818f4516a3ffec1f9ffc9680
I20260811 10:11:16.952150 10268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:16.952805 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:16.946431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.22.177.62:37249)
0811 10:11:16.946690 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:16.946698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:16.946748 (+ 50us) server_negotiation.cc:407] Connection header received
0811 10:11:16.946900 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:16.946906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:16.946961 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:16.947102 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:16.947319 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.949108 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.950300 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:16.950894 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:16.951025 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:16.951871 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:16.951876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:16.951876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:16.952106 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:16.952151 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:16.952168 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:16.952232 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:16.952450 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 10:11:16.952538 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260811 10:11:16.964491 10268 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:16.965371 10268 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:16.969612 10268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: Generated new TSK 0
I20260811 10:11:16.969893 10268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:16.980986 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37249
--builtin_ntp_servers=127.22.177.20:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:17.075357 10279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:17.075587 10279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:17.075609 10279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:17.076860 10279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:17.076915 10279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:17.076942 10279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:11:17.078486 10279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:17.078763 10279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:17.079115 10279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:17.084643 10284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:17.084750 10285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:17.084831 10279 server_base.cc:1034] running on GCE node
W20260811 10:11:17.084643 10287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:17.085086 10279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:17.085429 10279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:17.086588 10279 hybrid_clock.cc:648] HybridClock initialized: now 1786443077086549 us; error 56 us; skew 500 ppm
I20260811 10:11:17.087994 10279 webserver.cc:460] Webserver started at http://127.22.177.1:34121/ using document root <none> and password file <none>
I20260811 10:11:17.088274 10279 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:17.088333 10279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:17.088495 10279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:17.089658 10279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "f735c4d75c5d46248dbd4c65181b6f80"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "857d17dd4197a1cce64377a18e9b8a18"
server_key_iv: "c9af1d7b11497b59984e517c26738b1f"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.090063 10279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "f735c4d75c5d46248dbd4c65181b6f80"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "857d17dd4197a1cce64377a18e9b8a18"
server_key_iv: "c9af1d7b11497b59984e517c26738b1f"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.091456 10279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:17.092259 10293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.092447 10279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:17.092566 10279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "f735c4d75c5d46248dbd4c65181b6f80"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "857d17dd4197a1cce64377a18e9b8a18"
server_key_iv: "c9af1d7b11497b59984e517c26738b1f"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.092645 10279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:17.125404 10279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:17.125797 10279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:17.125959 10279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:17.126224 10279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:17.126618 10279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:17.126677 10279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.126720 10279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:17.126749 10279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.133226 10279 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:34789
I20260811 10:11:17.133306 10421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:34789 every 8 connection(s)
I20260811 10:11:17.133702 10279 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:11:17.138746 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 10279
I20260811 10:11:17.138883 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:11:17.139117 23236 external_mini_cluster.cc:1428] Setting key af573df76bbd8be6cc695d8ba4b1a032
I20260811 10:11:17.139430 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.134578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:47723 (local address 127.22.177.62:37249)
0811 10:11:17.134680 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.134682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.134875 (+ 193us) server_negotiation.cc:407] Connection header received
0811 10:11:17.135349 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.135350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.135365 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.135445 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:17.136129 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.136819 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.137899 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.138102 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.138161 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.139085 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:17.139086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:17.139086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:17.139101 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:17.139118 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:17.139122 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:17.139178 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.139326 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.139372 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 10:11:17.139813 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.134529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37249 (local address 127.22.177.1:47723)
0811 10:11:17.134711 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.134752 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.135249 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.135496 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.135499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.135506 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:17.136024 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.136028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.136918 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.136919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.137819 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.137822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.138183 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.138184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.138294 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.138840 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:17.138841 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:17.138888 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:17.138919 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:17.139201 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:17.139210 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:17.139213 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.139544 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.139644 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260811 10:11:17.140307 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37249
--builtin_ntp_servers=127.22.177.20:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:11:17.140419 10422 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37249
I20260811 10:11:17.140553 10422 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:17.140753 10422 heartbeater.cc:511] Master 127.22.177.62:37249 requested a full tablet report, sending...
I20260811 10:11:17.141360 10213 ts_manager.cc:194] Registered new tserver with Master: f735c4d75c5d46248dbd4c65181b6f80 (127.22.177.1:34789)
I20260811 10:11:17.142355 10213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.1:47723
W20260811 10:11:17.229324 10425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:17.229584 10425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:17.229657 10425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:17.230960 10425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:17.231045 10425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:17.231091 10425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:11:17.232586 10425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:17.232890 10425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:17.233237 10425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:17.239130 10431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:17.239169 10425 server_base.cc:1034] running on GCE node
W20260811 10:11:17.239149 10435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:17.239149 10433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:17.239805 10425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:17.240090 10425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:17.241278 10425 hybrid_clock.cc:648] HybridClock initialized: now 1786443077241279 us; error 45 us; skew 500 ppm
I20260811 10:11:17.242875 10425 webserver.cc:460] Webserver started at http://127.22.177.2:43051/ using document root <none> and password file <none>
I20260811 10:11:17.243160 10425 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:17.243211 10425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:17.243359 10425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:17.244469 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "d2f56d39bec6495c9d691c879b2b91c0"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "e71402d8fd0d90e32f4235a5285983df"
server_key_iv: "d4a54eecf186669392a9ab46befe5302"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.244845 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "d2f56d39bec6495c9d691c879b2b91c0"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "e71402d8fd0d90e32f4235a5285983df"
server_key_iv: "d4a54eecf186669392a9ab46befe5302"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.246322 10425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:17.247342 10443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.247560 10425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:17.247673 10425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "d2f56d39bec6495c9d691c879b2b91c0"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "e71402d8fd0d90e32f4235a5285983df"
server_key_iv: "d4a54eecf186669392a9ab46befe5302"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.247761 10425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:17.262257 10425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:17.262619 10425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:17.262781 10425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:17.263109 10425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:17.263469 10425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:17.263527 10425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.263571 10425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:17.263603 10425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.270094 10425 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:35673
I20260811 10:11:17.270141 10576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:35673 every 8 connection(s)
I20260811 10:11:17.270576 10425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:11:17.276312 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.271421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:41297 (local address 127.22.177.62:37249)
0811 10:11:17.271494 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.271496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.271720 (+ 224us) server_negotiation.cc:407] Connection header received
0811 10:11:17.272301 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.272302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.272316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.272361 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:17.273078 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.273640 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.274649 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.274855 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.274937 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.275980 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:17.275982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:17.275982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:17.275988 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:17.276012 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:17.276017 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:17.276081 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.276180 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.276216 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:11:17.276557 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.271357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37249 (local address 127.22.177.2:41297)
0811 10:11:17.271594 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.271630 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.272162 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.272438 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.272441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.272450 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:17.272971 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.272975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.273746 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.273747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.274546 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.274549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.274947 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.274949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.275091 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.275821 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:17.275822 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:17.275852 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:17.275885 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:17.276083 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:17.276090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:17.276093 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.276182 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.276297 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":82,"threads_started":1}
I20260811 10:11:17.277276 10577 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37249
I20260811 10:11:17.277388 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 10425
I20260811 10:11:17.277441 10577 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:17.277501 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:11:17.277729 10577 heartbeater.cc:511] Master 127.22.177.62:37249 requested a full tablet report, sending...
I20260811 10:11:17.277853 23236 external_mini_cluster.cc:1428] Setting key cd3e28f2d727bac905681f8f0273a9f5
I20260811 10:11:17.278249 10213 ts_manager.cc:194] Registered new tserver with Master: d2f56d39bec6495c9d691c879b2b91c0 (127.22.177.2:35673)
I20260811 10:11:17.278795 10213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.2:41297
I20260811 10:11:17.279217 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37249
--builtin_ntp_servers=127.22.177.20:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:17.367187 10580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:17.367416 10580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:17.367439 10580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:17.368666 10580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:17.368716 10580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:17.368800 10580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:11:17.370313 10580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:37249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:17.370566 10580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:17.370952 10580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:11:17.376569 10580 server_base.cc:1034] running on GCE node
W20260811 10:11:17.376704 10586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:17.376580 10587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:17.376588 10589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:17.377131 10580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:17.377471 10580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:17.378672 10580 hybrid_clock.cc:648] HybridClock initialized: now 1786443077378617 us; error 90 us; skew 500 ppm
I20260811 10:11:17.380218 10580 webserver.cc:460] Webserver started at http://127.22.177.3:43129/ using document root <none> and password file <none>
I20260811 10:11:17.380579 10580 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:17.380661 10580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:17.380821 10580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:17.382036 10580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "bfc67d06e08a4e258b7def3192b016b1"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "bdaeca50132c32907969d88e6e827f8e"
server_key_iv: "54b322c1a5bca071a204aa4bd563d346"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.382424 10580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "bfc67d06e08a4e258b7def3192b016b1"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "bdaeca50132c32907969d88e6e827f8e"
server_key_iv: "54b322c1a5bca071a204aa4bd563d346"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.383939 10580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:17.384899 10595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.385075 10580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:17.385195 10580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "bfc67d06e08a4e258b7def3192b016b1"
format_stamp: "Formatted at 2026-08-11 10:11:17 on dist-test-slave-g350"
server_key: "bdaeca50132c32907969d88e6e827f8e"
server_key_iv: "54b322c1a5bca071a204aa4bd563d346"
server_key_version: "encryptionkey@0"
I20260811 10:11:17.385345 10580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:17.404102 10580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:17.404487 10580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:17.404654 10580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:17.404933 10580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:17.405314 10580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:17.405372 10580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.405416 10580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:17.405445 10580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:17.411773 10580 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:42945
I20260811 10:11:17.411819 10729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:42945 every 8 connection(s)
I20260811 10:11:17.412186 10580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:11:17.415846 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 10580
I20260811 10:11:17.416500 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:11:17.416715 23236 external_mini_cluster.cc:1428] Setting key 9784e07a390618ba5343f2a444a855a4
I20260811 10:11:17.417732 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.412984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:39053 (local address 127.22.177.62:37249)
0811 10:11:17.413053 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.413055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.413269 (+ 214us) server_negotiation.cc:407] Connection header received
0811 10:11:17.413877 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.413878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.413890 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.413946 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:17.414637 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.415158 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.416147 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.416362 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.416441 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.417425 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:17.417427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:17.417427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:17.417433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:17.417447 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:17.417451 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:17.417521 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.417615 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.417649 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:11:17.417990 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.412976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37249 (local address 127.22.177.3:39053)
0811 10:11:17.413149 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.413184 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.413761 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.414015 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.414019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.414031 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:17.414541 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.414545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.415262 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.415263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.416052 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.416054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.416454 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.416455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.416633 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.417288 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:17.417289 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:17.417326 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:17.417354 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:17.417523 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:17.417530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:17.417533 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.417630 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.417731 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260811 10:11:17.418542 10730 heartbeater.cc:347] Connected to a master server at 127.22.177.62:37249
I20260811 10:11:17.418648 10730 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:17.418834 10730 heartbeater.cc:511] Master 127.22.177.62:37249 requested a full tablet report, sending...
I20260811 10:11:17.419183 10213 ts_manager.cc:194] Registered new tserver with Master: bfc67d06e08a4e258b7def3192b016b1 (127.22.177.3:42945)
I20260811 10:11:17.419595 10213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.3:39053
I20260811 10:11:17.429044 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:11:17.433889 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.430739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.22.177.62:37249)
0811 10:11:17.430821 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.430823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.430947 (+ 124us) server_negotiation.cc:407] Connection header received
0811 10:11:17.431010 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.431011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.431023 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.431086 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:17.431327 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.431811 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.432537 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.432717 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.432773 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.433641 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:17.433642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:17.433643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:17.433648 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:17.433669 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:17.433674 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:17.433719 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.433805 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.433829 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 10:11:17.435763 10213 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:11:17.437309 10213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:11:17.444177 10738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.439767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:34789 (local address 127.0.0.1:53140)
0811 10:11:17.440178 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.440222 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.440486 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.440740 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.440741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.440743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:17.440775 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.440776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.441687 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.441689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.442920 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.442923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.443856 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.443857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.443945 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.443960 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.444032 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.444089 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":107,"threads_started":1}
I20260811 10:11:17.444202 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.439862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53140 (local address 127.22.177.1:34789)
0811 10:11:17.440073 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.440080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.440455 (+ 375us) server_negotiation.cc:407] Connection header received
0811 10:11:17.440580 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.440582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.440597 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.440649 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:17.440861 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.441550 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.442997 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.443765 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.443847 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.443949 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.444056 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.444100 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260811 10:11:17.444370 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.440005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.22.177.3:42945)
0811 10:11:17.440169 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.440172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.440407 (+ 235us) server_negotiation.cc:407] Connection header received
0811 10:11:17.440453 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.440456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.440472 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.440542 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:17.440788 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.441953 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.443160 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.443852 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.443912 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.443986 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.444245 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.444298 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260811 10:11:17.444577 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.439819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:42945 (local address 127.0.0.1:60556)
0811 10:11:17.440198 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.440232 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.440345 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.440622 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.440623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.440625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:17.440679 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.440680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.442064 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.442065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.443078 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.443080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.444027 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.444029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.444095 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.444099 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.444154 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.444197 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260811 10:11:17.445125 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.440029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35673 (local address 127.0.0.1:42498)
0811 10:11:17.440224 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.440234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.440306 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.440572 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.440575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.440578 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:17.440658 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.440660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.443171 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.443171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.443879 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.443881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.444881 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.444882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.444944 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.444949 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.445005 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.445046 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260811 10:11:17.445138 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.440128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.22.177.2:35673)
0811 10:11:17.440327 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.440331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.440347 (+ 16us) server_negotiation.cc:407] Connection header received
0811 10:11:17.440393 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.440396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.440415 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.440495 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:17.440770 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.441966 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.443977 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.444804 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.444878 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.444962 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.445017 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.445063 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260811 10:11:17.445276 10348 tablet_service.cc:1467] Processing CreateTablet for tablet ced837b9388548d9b317a9da54670bd0 (DEFAULT_TABLE table=test-table [id=ba9727dd18df40168a76bffaa056839e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:17.445396 10649 tablet_service.cc:1467] Processing CreateTablet for tablet ced837b9388548d9b317a9da54670bd0 (DEFAULT_TABLE table=test-table [id=ba9727dd18df40168a76bffaa056839e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:17.445750 10348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced837b9388548d9b317a9da54670bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:17.445811 10649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced837b9388548d9b317a9da54670bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:17.445961 10498 tablet_service.cc:1467] Processing CreateTablet for tablet ced837b9388548d9b317a9da54670bd0 (DEFAULT_TABLE table=test-table [id=ba9727dd18df40168a76bffaa056839e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:17.446277 10498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced837b9388548d9b317a9da54670bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:17.448904 10745 tablet_bootstrap.cc:492] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80: Bootstrap starting.
I20260811 10:11:17.449666 10745 tablet_bootstrap.cc:654] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:17.449998 10746 tablet_bootstrap.cc:492] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Bootstrap starting.
I20260811 10:11:17.450095 10747 tablet_bootstrap.cc:492] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: Bootstrap starting.
I20260811 10:11:17.450104 10745 log.cc:824] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:17.450704 10746 tablet_bootstrap.cc:654] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:17.450758 10747 tablet_bootstrap.cc:654] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:17.451010 10745 tablet_bootstrap.cc:492] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80: No bootstrap required, opened a new log
I20260811 10:11:17.451087 10745 ts_tablet_manager.cc:1397] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:11:17.451102 10747 log.cc:824] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:17.451123 10746 log.cc:824] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:17.451802 10746 tablet_bootstrap.cc:492] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: No bootstrap required, opened a new log
I20260811 10:11:17.451856 10746 ts_tablet_manager.cc:1397] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:11:17.453317 10745 raft_consensus.cc:348] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.453480 10745 raft_consensus.cc:374] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:17.453528 10745 raft_consensus.cc:729] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f735c4d75c5d46248dbd4c65181b6f80, State: Initialized, Role: FOLLOWER
I20260811 10:11:17.453681 10745 consensus_queue.cc:260] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.454007 10745 ts_tablet_manager.cc:1428] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 10:11:17.453945 10746 raft_consensus.cc:348] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.454066 10422 heartbeater.cc:503] Master 127.22.177.62:37249 was elected leader, sending a full tablet report...
I20260811 10:11:17.454118 10746 raft_consensus.cc:374] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:17.454197 10746 raft_consensus.cc:729] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc67d06e08a4e258b7def3192b016b1, State: Initialized, Role: FOLLOWER
I20260811 10:11:17.454481 10746 consensus_queue.cc:260] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.454826 10747 tablet_bootstrap.cc:492] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: No bootstrap required, opened a new log
I20260811 10:11:17.454869 10746 ts_tablet_manager.cc:1428] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:17.454955 10747 ts_tablet_manager.cc:1397] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 10:11:17.455024 10730 heartbeater.cc:503] Master 127.22.177.62:37249 was elected leader, sending a full tablet report...
I20260811 10:11:17.457139 10747 raft_consensus.cc:348] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.457276 10747 raft_consensus.cc:374] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:17.457309 10747 raft_consensus.cc:729] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f56d39bec6495c9d691c879b2b91c0, State: Initialized, Role: FOLLOWER
I20260811 10:11:17.457439 10747 consensus_queue.cc:260] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.457731 10747 ts_tablet_manager.cc:1428] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:11:17.457808 10577 heartbeater.cc:503] Master 127.22.177.62:37249 was elected leader, sending a full tablet report...
W20260811 10:11:17.521204 10578 tablet.cc:2369] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:11:17.533982 10754 raft_consensus.cc:482] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:17.534175 10754 raft_consensus.cc:504] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.534683 10754 leader_election.cc:288] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f735c4d75c5d46248dbd4c65181b6f80 (127.22.177.1:34789), bfc67d06e08a4e258b7def3192b016b1 (127.22.177.3:42945)
I20260811 10:11:17.539052 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.535067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:59891 (local address 127.22.177.1:34789)
0811 10:11:17.535121 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.535123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.535299 (+ 176us) server_negotiation.cc:407] Connection header received
0811 10:11:17.535360 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.535361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.535375 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.535429 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:17.535595 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.536363 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.538050 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.538661 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.538728 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.538737 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.538927 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.538978 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:11:17.539052 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.535053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:38889 (local address 127.22.177.3:42945)
0811 10:11:17.535099 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.535102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.535112 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:17.535171 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.535172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.535181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.535330 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:17.535593 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.536361 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.538045 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.538601 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.538688 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.538698 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.538927 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.538976 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:17.539039 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.534933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:42945 (local address 127.22.177.2:38889)
0811 10:11:17.535011 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.535026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.535099 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.535399 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.535399 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.535401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:17.535499 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.535501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.536458 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.536459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.537815 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.537818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.538718 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.538720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.538800 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.538805 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.538871 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.538922 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":44}
I20260811 10:11:17.539045 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.534933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:34789 (local address 127.22.177.2:59891)
0811 10:11:17.535206 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.535220 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:17.535276 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:17.535448 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:17.535449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:17.535450 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:17.535516 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.535518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.536457 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.536459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.537815 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:17.537818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.538745 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.538746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:17.538816 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.538819 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:17.538892 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 10:11:17.538924 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":181,"thread_start_us":108,"threads_started":1}
I20260811 10:11:17.539464 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced837b9388548d9b317a9da54670bd0" candidate_uuid: "d2f56d39bec6495c9d691c879b2b91c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc67d06e08a4e258b7def3192b016b1" is_pre_election: true
I20260811 10:11:17.539502 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced837b9388548d9b317a9da54670bd0" candidate_uuid: "d2f56d39bec6495c9d691c879b2b91c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f735c4d75c5d46248dbd4c65181b6f80" is_pre_election: true
I20260811 10:11:17.539647 10374 raft_consensus.cc:2454] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f56d39bec6495c9d691c879b2b91c0 in term 0.
I20260811 10:11:17.539647 10675 raft_consensus.cc:2454] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f56d39bec6495c9d691c879b2b91c0 in term 0.
I20260811 10:11:17.539934 10446 leader_election.cc:302] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2f56d39bec6495c9d691c879b2b91c0, f735c4d75c5d46248dbd4c65181b6f80; no voters:
I20260811 10:11:17.540145 10754 raft_consensus.cc:2781] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:11:17.540289 10754 raft_consensus.cc:482] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:11:17.540325 10754 raft_consensus.cc:3037] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:17.541086 10754 raft_consensus.cc:504] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.541270 10754 leader_election.cc:288] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [CANDIDATE]: Term 1 election: Requested vote from peers f735c4d75c5d46248dbd4c65181b6f80 (127.22.177.1:34789), bfc67d06e08a4e258b7def3192b016b1 (127.22.177.3:42945)
I20260811 10:11:17.541482 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced837b9388548d9b317a9da54670bd0" candidate_uuid: "d2f56d39bec6495c9d691c879b2b91c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc67d06e08a4e258b7def3192b016b1"
I20260811 10:11:17.541482 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced837b9388548d9b317a9da54670bd0" candidate_uuid: "d2f56d39bec6495c9d691c879b2b91c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f735c4d75c5d46248dbd4c65181b6f80"
I20260811 10:11:17.541575 10675 raft_consensus.cc:3037] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:17.541575 10374 raft_consensus.cc:3037] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:17.542558 10675 raft_consensus.cc:2454] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f56d39bec6495c9d691c879b2b91c0 in term 1.
I20260811 10:11:17.542558 10374 raft_consensus.cc:2454] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f56d39bec6495c9d691c879b2b91c0 in term 1.
I20260811 10:11:17.542804 10446 leader_election.cc:302] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2f56d39bec6495c9d691c879b2b91c0, f735c4d75c5d46248dbd4c65181b6f80; no voters:
I20260811 10:11:17.542950 10754 raft_consensus.cc:2781] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:17.543175 10754 raft_consensus.cc:686] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 1 LEADER]: Becoming Leader. State: Replica: d2f56d39bec6495c9d691c879b2b91c0, State: Running, Role: LEADER
I20260811 10:11:17.543345 10754 consensus_queue.cc:237] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } }
I20260811 10:11:17.544489 10212 catalog_manager.cc:5495] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2f56d39bec6495c9d691c879b2b91c0 (127.22.177.2). New cstate: current_term: 1 leader_uuid: "d2f56d39bec6495c9d691c879b2b91c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2f56d39bec6495c9d691c879b2b91c0" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 35673 } health_report { overall_health: HEALTHY } } }
I20260811 10:11:17.563935 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.561533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.22.177.2:35673)
0811 10:11:17.561650 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:17.561652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:17.561662 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:17.561711 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:17.561711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:17.561732 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:17.561794 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:17.561931 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.562406 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.563136 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:17.563323 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:17.563376 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:17.563508 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:17.563609 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:17.563678 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:17.563789 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 10:11:17.563807 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:11:17.565860 10374 raft_consensus.cc:1261] T ced837b9388548d9b317a9da54670bd0 P f735c4d75c5d46248dbd4c65181b6f80 [term 1 FOLLOWER]: Refusing update from remote peer d2f56d39bec6495c9d691c879b2b91c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:17.565858 10675 raft_consensus.cc:1261] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 1 FOLLOWER]: Refusing update from remote peer d2f56d39bec6495c9d691c879b2b91c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:17.566246 10756 consensus_queue.cc:1035] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc67d06e08a4e258b7def3192b016b1" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 42945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:17.566376 10754 consensus_queue.cc:1035] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f735c4d75c5d46248dbd4c65181b6f80" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:17.569007 10761 mvcc.cc:205] Tried to move back new op lower bound from 7317270845706854400 to 7317270845618262016. Current Snapshot: MvccSnapshot[applied={T|T < 7317270845706854400}]
I20260811 10:11:17.569176 10759 mvcc.cc:205] Tried to move back new op lower bound from 7317270845706854400 to 7317270845618262016. Current Snapshot: MvccSnapshot[applied={T|T < 7317270845706854400}]
I20260811 10:11:17.570437 10760 mvcc.cc:205] Tried to move back new op lower bound from 7317270845706854400 to 7317270845618262016. Current Snapshot: MvccSnapshot[applied={T|T < 7317270845706854400 or (T in {7317270845706854400})}]
I20260811 10:11:17.571314 10213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:17.571416 10213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:17.572237 10213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfe76da64edb4619bffe3581f36417fc: Sending DeleteTablet for 3 replicas of tablet ced837b9388548d9b317a9da54670bd0
I20260811 10:11:17.572629 10348 tablet_service.cc:1514] Processing DeleteTablet for tablet ced837b9388548d9b317a9da54670bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:17 UTC) from {username='slave'} at 127.0.0.1:53140
I20260811 10:11:17.572634 10649 tablet_service.cc:1514] Processing DeleteTablet for tablet ced837b9388548d9b317a9da54670bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:17 UTC) from {username='slave'} at 127.0.0.1:60556
I20260811 10:11:17.572875 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 10279
I20260811 10:11:17.572938 10498 tablet_service.cc:1514] Processing DeleteTablet for tablet ced837b9388548d9b317a9da54670bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:17 UTC) from {username='slave'} at 127.0.0.1:42498
I20260811 10:11:17.573410 10770 tablet_replica.cc:330] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: stopping tablet replica
I20260811 10:11:17.573516 10770 raft_consensus.cc:2229] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:17.573650 10770 raft_consensus.cc:2258] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:17.574550 10770 ts_tablet_manager.cc:1905] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:17.577054 10770 ts_tablet_manager.cc:1918] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:17.577140 10770 log.cc:1196] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/ced837b9388548d9b317a9da54670bd0
I20260811 10:11:17.577414 10770 ts_tablet_manager.cc:1939] T ced837b9388548d9b317a9da54670bd0 P bfc67d06e08a4e258b7def3192b016b1: Deleting consensus metadata
I20260811 10:11:17.577879 10193 catalog_manager.cc:4841] TS bfc67d06e08a4e258b7def3192b016b1 (127.22.177.3:42945): tablet ced837b9388548d9b317a9da54670bd0 (table test-table [id=ba9727dd18df40168a76bffaa056839e]) successfully deleted
W20260811 10:11:17.582830 10195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:34789 (error 108)
I20260811 10:11:17.582917 10771 tablet_replica.cc:330] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0: stopping tablet replica
I20260811 10:11:17.583053 10771 raft_consensus.cc:2229] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:11:17.583189 10771 raft_consensus.cc:2258] T ced837b9388548d9b317a9da54670bd0 P d2f56d39bec6495c9d691c879b2b91c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:17.583513 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.583366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:34789 (local address 127.0.0.1:53148)
0811 10:11:17.583425 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.583467 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:34789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 10:11:17.583585 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 10425
W20260811 10:11:17.585026 10195 catalog_manager.cc:4584] TS f735c4d75c5d46248dbd4c65181b6f80 (127.22.177.1:34789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ced837b9388548d9b317a9da54670bd0: Network error: Client connection negotiation failed: client connection to 127.22.177.1:34789: connect: Connection refused (error 111)
I20260811 10:11:17.590870 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 10580
I20260811 10:11:17.590963 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:17.590813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:35673 (local address 127.0.0.1:42518)
0811 10:11:17.590884 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:17.590920 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.2:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 10:11:17.598814 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 10176
2026-08-11T10:11:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (820 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:17 dist-test-slave-g350 krb5kdc[10775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:17 dist-test-slave-g350 krb5kdc[10775](info): set up 2 sockets
Aug 11 10:11:17 dist-test-slave-g350 krb5kdc[10775](info): commencing operation
krb5kdc: starting...
W20260811 10:11:18.750602 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:18 dist-test-slave-g350 krb5kdc[10775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:18.898773 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:41799
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:18.984246 10789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:18.984469 10789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:18.984493 10789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:18.985637 10789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:18.985680 10789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:18.985694 10789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:18.985707 10789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:18.985719 10789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:18.987198 10789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:41799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:18.987421 10789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:18.987609 10789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:18.992908 10796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:18.992949 10798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:18.993119 10789 server_base.cc:1034] running on GCE node
W20260811 10:11:18.992908 10800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:18.993535 10789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:18.993881 10789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:18.995064 10789 hybrid_clock.cc:648] HybridClock initialized: now 1786443078995032 us; error 52 us; skew 500 ppm
Aug 11 10:11:18 dist-test-slave-g350 krb5kdc[10775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443078, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:19.003091 10789 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:19.003546 10789 webserver.cc:460] Webserver started at http://127.22.177.62:41171/ using document root <none> and password file <none>
I20260811 10:11:19.003751 10789 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:19.003794 10789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:19.003892 10789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:19.004985 10789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "b575e663d235410c9c227a889885566a"
format_stamp: "Formatted at 2026-08-11 10:11:18 on dist-test-slave-g350"
server_key: "6f26f6c40ce072a37ab74b8be534793f"
server_key_iv: "fe43731e5fefc29763af46047e85a1ad"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.005318 10789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "b575e663d235410c9c227a889885566a"
format_stamp: "Formatted at 2026-08-11 10:11:18 on dist-test-slave-g350"
server_key: "6f26f6c40ce072a37ab74b8be534793f"
server_key_iv: "fe43731e5fefc29763af46047e85a1ad"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.006834 10789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:19.007722 10809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.007927 10789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:19.008064 10789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "b575e663d235410c9c227a889885566a"
format_stamp: "Formatted at 2026-08-11 10:11:18 on dist-test-slave-g350"
server_key: "6f26f6c40ce072a37ab74b8be534793f"
server_key_iv: "fe43731e5fefc29763af46047e85a1ad"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.008147 10789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:19.018641 10789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:19.020684 10789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:19.020870 10789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:19.025365 10789 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:41799
I20260811 10:11:19.025400 10869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:41799 every 8 connection(s)
I20260811 10:11:19.025822 10789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:19.026773 10871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:19.029140 10871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a: Bootstrap starting.
I20260811 10:11:19.029832 10871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:19.030129 10871 log.cc:824] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:19.030938 10871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a: No bootstrap required, opened a new log
I20260811 10:11:19.032651 10871 raft_consensus.cc:348] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b575e663d235410c9c227a889885566a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 41799 } }
I20260811 10:11:19.032804 10871 raft_consensus.cc:374] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:19.032828 10871 raft_consensus.cc:729] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b575e663d235410c9c227a889885566a, State: Initialized, Role: FOLLOWER
I20260811 10:11:19.033005 10871 consensus_queue.cc:260] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b575e663d235410c9c227a889885566a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 41799 } }
I20260811 10:11:19.033077 10871 raft_consensus.cc:388] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:19.033098 10871 raft_consensus.cc:482] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:19.033164 10871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:19.033994 10871 raft_consensus.cc:504] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b575e663d235410c9c227a889885566a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 41799 } }
I20260811 10:11:19.034129 10871 leader_election.cc:302] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b575e663d235410c9c227a889885566a; no voters:
I20260811 10:11:19.034335 10871 leader_election.cc:288] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:19.034421 10874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:19.034639 10871 sys_catalog.cc:564] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:19.034669 10874 raft_consensus.cc:686] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [term 1 LEADER]: Becoming Leader. State: Replica: b575e663d235410c9c227a889885566a, State: Running, Role: LEADER
I20260811 10:11:19.034770 10874 consensus_queue.cc:237] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b575e663d235410c9c227a889885566a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 41799 } }
I20260811 10:11:19.035202 10877 sys_catalog.cc:455] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b575e663d235410c9c227a889885566a. Latest consensus state: current_term: 1 leader_uuid: "b575e663d235410c9c227a889885566a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b575e663d235410c9c227a889885566a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 41799 } } }
I20260811 10:11:19.035290 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 10789
I20260811 10:11:19.035409 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:19.035377 10875 sys_catalog.cc:455] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b575e663d235410c9c227a889885566a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b575e663d235410c9c227a889885566a" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 41799 } } }
I20260811 10:11:19.035454 10875 sys_catalog.cc:458] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:19.035375 10877 sys_catalog.cc:458] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:19.035647 23236 external_mini_cluster.cc:1428] Setting key 450cdcee26ca5889509d61a1cf1e5315
I20260811 10:11:19.035852 10884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:19.036392 10884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:19.038091 10884 catalog_manager.cc:1347] Generated new cluster ID: 56fbd3e91c0f4d0fb3e90c98e531f28b
I20260811 10:11:19.038148 10884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:19.045392 10884 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:19.046104 10884 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:19.049221 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.036350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.22.177.62:41799)
0811 10:11:19.037763 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.037772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.037806 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:11:19.037940 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.037947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.038015 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.038166 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.038992 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.040384 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.041482 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.042104 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.042210 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.045255 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.045260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.045260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:19.045274 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.047460 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.047898 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.047902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.047902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.048018 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.048235 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.048235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.048236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.048534 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.048686 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.048803 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.048964 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1351,"thread_start_us":71,"threads_started":1}
I20260811 10:11:19.053452 10884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a: Generated new TSK 0
I20260811 10:11:19.053699 10884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:11:19.085865 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:41799
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:19.171238 10899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:19.171478 10899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:19.171518 10899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:19.173224 10899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:19.173287 10899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:19.173332 10899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:11:19.174773 10899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:19.175031 10899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:19.175271 10899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:19.180720 10905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.180842 10899 server_base.cc:1034] running on GCE node
W20260811 10:11:19.180694 10907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.180784 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.181138 10899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:19.181432 10899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:19.182582 10899 hybrid_clock.cc:648] HybridClock initialized: now 1786443079182553 us; error 41 us; skew 500 ppm
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:19.186424 10899 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:11:19.187075 10899 webserver.cc:460] Webserver started at http://127.22.177.1:32821/ using document root <none> and password file <none>
I20260811 10:11:19.187384 10899 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:19.187449 10899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:19.187593 10899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:19.189215 10899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "24b33690fc65448e9f9d92c460fb70c4"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "9e0f15283578f60d1bef9f2c03293ab5"
server_key_iv: "450fff02bf63642667d6d827ac07aaa9"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.189668 10899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "24b33690fc65448e9f9d92c460fb70c4"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "9e0f15283578f60d1bef9f2c03293ab5"
server_key_iv: "450fff02bf63642667d6d827ac07aaa9"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.191303 10899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:19.192376 10914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.192581 10899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:11:19.192677 10899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "24b33690fc65448e9f9d92c460fb70c4"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "9e0f15283578f60d1bef9f2c03293ab5"
server_key_iv: "450fff02bf63642667d6d827ac07aaa9"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.192757 10899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:19.217692 10899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:19.220016 10899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:19.220237 10899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:19.220546 10899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:19.220986 10899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:19.221050 10899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.221102 10899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:19.221167 10899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.228675 10899 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:45335
I20260811 10:11:19.228742 11046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:45335 every 8 connection(s)
I20260811 10:11:19.229111 10899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:11:19.233029 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 10899
I20260811 10:11:19.233204 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:11:19.233477 23236 external_mini_cluster.cc:1428] Setting key b4253f021f52dc2731c5b5062903109f
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:19.241250 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.229884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:55309 (local address 127.22.177.62:41799)
0811 10:11:19.229972 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.229975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.230192 (+ 217us) server_negotiation.cc:407] Connection header received
0811 10:11:19.230790 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.230791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.230802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.230867 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.232014 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.232547 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.233955 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.234273 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.234357 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.237435 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.237437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.237437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:19.237447 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.239406 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.239850 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.239851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.239852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.239922 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.240246 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.240247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.240248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.240418 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.240500 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.240996 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.241108 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 10:11:19.241425 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.229837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:41799 (local address 127.22.177.1:55309)
0811 10:11:19.230050 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.230093 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.230659 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.230961 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.230965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.231691 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:19.231916 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.231921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.232670 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.232671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.233846 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.233849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.234378 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.234379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.234547 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.235167 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:19.235180 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:19.237327 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:19.239512 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:19.239518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:19.239520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:19.239755 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:19.240017 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:19.240018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:19.240018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:19.240150 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:19.240499 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:19.240505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:19.240821 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.240943 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.241112 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260811 10:11:19.242142 11049 heartbeater.cc:347] Connected to a master server at 127.22.177.62:41799
I20260811 10:11:19.242298 11049 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:19.242547 11049 heartbeater.cc:511] Master 127.22.177.62:41799 requested a full tablet report, sending...
I20260811 10:11:19.243307 10830 ts_manager.cc:194] Registered new tserver with Master: 24b33690fc65448e9f9d92c460fb70c4 (127.22.177.1:45335)
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
I20260811 10:11:19.244326 10830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:55309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:11:19.266213 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:41799
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:19.357348 11055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:19.357625 11055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:19.357651 11055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:19.358793 11055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:19.358834 11055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:19.358901 11055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:11:19.360334 11055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:19.360528 11055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:19.360720 11055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:19.366267 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.366271 11061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.366495 11055 server_base.cc:1034] running on GCE node
W20260811 10:11:19.366402 11060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.366837 11055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:19.367147 11055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:19.368300 11055 hybrid_clock.cc:648] HybridClock initialized: now 1786443079368285 us; error 48 us; skew 500 ppm
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:19.371011 11055 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:11:19.371510 11055 webserver.cc:460] Webserver started at http://127.22.177.2:40549/ using document root <none> and password file <none>
I20260811 10:11:19.371706 11055 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:19.371747 11055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:19.371843 11055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:19.372892 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "640316d070d3410d84344490a46e22ac"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "2af438a6f4cdc766ae2c4d557c69e777"
server_key_iv: "2a3e579b4411c5e03563f6522e44712e"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.373204 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "640316d070d3410d84344490a46e22ac"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "2af438a6f4cdc766ae2c4d557c69e777"
server_key_iv: "2a3e579b4411c5e03563f6522e44712e"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.374644 11055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:19.375444 11073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.375651 11055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:19.375751 11055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "640316d070d3410d84344490a46e22ac"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "2af438a6f4cdc766ae2c4d557c69e777"
server_key_iv: "2a3e579b4411c5e03563f6522e44712e"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.375826 11055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:19.390110 11055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:19.392606 11055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:19.392767 11055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:19.393015 11055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:19.393340 11055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:19.393375 11055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.393399 11055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:19.393445 11055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.399732 11055 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:39529
I20260811 10:11:19.399796 11199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:39529 every 8 connection(s)
I20260811 10:11:19.400168 11055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:11:19.402678 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 11055
I20260811 10:11:19.402784 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:11:19.403008 23236 external_mini_cluster.cc:1428] Setting key 00de128cdee7ed4c8406677f5643cd5d
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:19.411724 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.401004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:39085 (local address 127.22.177.62:41799)
0811 10:11:19.401101 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.401104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.401280 (+ 176us) server_negotiation.cc:407] Connection header received
0811 10:11:19.401863 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.401864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.401876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.401945 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.403166 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.403708 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.405201 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.405415 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.405481 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.408250 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.408252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.408252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:19.408262 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.409959 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.410567 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.410569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.410574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.410640 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.410968 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.410969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.410970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.411094 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.411152 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.411567 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.411656 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 10:11:19.412022 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.400952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:41799 (local address 127.22.177.2:39085)
0811 10:11:19.401120 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.401176 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.401759 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.402018 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.402022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.402724 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:19.403003 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.403008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.403800 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.403802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.405096 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.405101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.405512 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.405513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.405711 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.406446 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:19.406455 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:19.408134 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:19.410050 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:19.410060 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:19.410062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:19.410456 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:19.410725 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:19.410726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:19.410726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:19.410892 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:19.411189 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:19.411197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:19.411396 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.411519 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.411730 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260811 10:11:19.412609 11201 heartbeater.cc:347] Connected to a master server at 127.22.177.62:41799
I20260811 10:11:19.412726 11201 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:19.412949 11201 heartbeater.cc:511] Master 127.22.177.62:41799 requested a full tablet report, sending...
I20260811 10:11:19.413525 10830 ts_manager.cc:194] Registered new tserver with Master: 640316d070d3410d84344490a46e22ac (127.22.177.2:39529)
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:11:19.414004 10830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:39085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:11:19.437119 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:41799
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:19.520843 11207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:19.521052 11207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:19.521075 11207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:19.522260 11207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:19.522312 11207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:19.522338 11207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:11:19.523859 11207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:19.524115 11207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:19.524371 11207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:19.529742 11212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.529819 11207 server_base.cc:1034] running on GCE node
W20260811 10:11:19.529755 11213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.529754 11215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.530164 11207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:19.530419 11207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:19.531594 11207 hybrid_clock.cc:648] HybridClock initialized: now 1786443079531593 us; error 43 us; skew 500 ppm
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:19.534228 11207 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:11:19.534765 11207 webserver.cc:460] Webserver started at http://127.22.177.3:32861/ using document root <none> and password file <none>
I20260811 10:11:19.535023 11207 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:19.535095 11207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:19.535256 11207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:19.536609 11207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "91152224c30844b2842ad94c78564d11"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "0a36ceea4e7565ceea76fa0fd995b4fe"
server_key_iv: "0aa3cf1d80d0835817ec074c9f7c0670"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.537037 11207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "91152224c30844b2842ad94c78564d11"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "0a36ceea4e7565ceea76fa0fd995b4fe"
server_key_iv: "0aa3cf1d80d0835817ec074c9f7c0670"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.538590 11207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 10:11:19.539487 11224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.539705 11207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:19.539811 11207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "91152224c30844b2842ad94c78564d11"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "0a36ceea4e7565ceea76fa0fd995b4fe"
server_key_iv: "0aa3cf1d80d0835817ec074c9f7c0670"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.539889 11207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:19.555720 11207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:19.558485 11207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:19.558655 11207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:19.558935 11207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:19.559274 11207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:19.559309 11207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.559355 11207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:19.559382 11207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.565852 11207 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:35111
I20260811 10:11:19.565938 11349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:35111 every 8 connection(s)
I20260811 10:11:19.566242 11207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:11:19 dist-test-slave-g350 krb5kdc[10775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:19.573268 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 11207
I20260811 10:11:19.573416 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:11:19.573661 23236 external_mini_cluster.cc:1428] Setting key 201ce4c0645f4fe4c05cd025f3bf9ed4
I20260811 10:11:19.576360 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.567024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:41435 (local address 127.22.177.62:41799)
0811 10:11:19.567113 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.567115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.567358 (+ 243us) server_negotiation.cc:407] Connection header received
0811 10:11:19.567900 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.567901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.567914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.567966 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.568936 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.569478 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.570539 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.570732 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.570815 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.573065 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.573067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.573067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:19.573082 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.574919 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.575330 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.575331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.575332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.575392 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.575661 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.575661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.575662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.575806 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.575898 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.576187 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.576268 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:11:19.576588 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.566990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:41799 (local address 127.22.177.3:41435)
0811 10:11:19.567189 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.567227 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.567783 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.568039 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.568042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.568627 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:19.568831 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.568836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.569583 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.569584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.570416 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.570418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.570836 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.570837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.570967 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.571518 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:19.571526 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:19.572951 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:19.574998 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:19.575005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:19.575008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:19.575246 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:19.575468 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:19.575469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:19.575469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:19.575584 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:19.575892 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:19.575899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:19.576045 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.576152 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.576327 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260811 10:11:19.577133 11350 heartbeater.cc:347] Connected to a master server at 127.22.177.62:41799
I20260811 10:11:19.577255 11350 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:19.577443 11350 heartbeater.cc:511] Master 127.22.177.62:41799 requested a full tablet report, sending...
I20260811 10:11:19.577876 10830 ts_manager.cc:194] Registered new tserver with Master: 91152224c30844b2842ad94c78564d11 (127.22.177.3:35111)
I20260811 10:11:19.578263 10830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:41435
I20260811 10:11:19.585994 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:11:19.594532 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.587550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.22.177.62:41799)
0811 10:11:19.587622 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.587624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.587786 (+ 162us) server_negotiation.cc:407] Connection header received
0811 10:11:19.587841 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.587841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.587851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.587905 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.588584 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.589226 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.590095 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.590282 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.590350 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.591812 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.591813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.591814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:19.591820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.593212 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.593712 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.593713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.593713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.593771 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:19.594001 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:19.594002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:19.594002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:19.594140 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.594221 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.594403 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.594466 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 10:11:19.596478 10830 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:11:19.598631 10830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:11:19.605407 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.601166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:45335 (local address 127.0.0.1:42428)
0811 10:11:19.601534 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.601581 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.601663 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.601929 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.601931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.601933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:19.602039 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.602041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.603070 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.603071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.604211 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.604214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.605118 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.605119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.605174 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.605183 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.605264 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.605323 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":5,"thread_start_us":73,"threads_started":1}
I20260811 10:11:19.605422 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.601286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.22.177.1:45335)
0811 10:11:19.601736 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.601740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.601750 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:19.601806 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.601808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.601826 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.601919 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:19.602108 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.602956 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.604317 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.604994 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.605031 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.605069 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.605295 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.605336 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260811 10:11:19.605929 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.601392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39529 (local address 127.0.0.1:50818)
0811 10:11:19.601782 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.601791 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.601827 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.602330 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.602331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.602334 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:19.602384 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.602386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.603641 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.603642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.604555 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.604557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.605683 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.605684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.605746 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.605750 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.605810 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.605854 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":74,"threads_started":1}
I20260811 10:11:19.606097 11365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.601169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:35111 (local address 127.0.0.1:44066)
0811 10:11:19.601688 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.601698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.601754 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.601932 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.601933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.601934 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:19.602039 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.602211 (+ 172us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.603534 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.603535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.604648 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.604651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.605865 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.605866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.605934 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.605938 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.605988 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.606034 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":30,"thread_start_us":84,"threads_started":1}
I20260811 10:11:19.606294 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.601506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.22.177.2:39529)
0811 10:11:19.602092 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.602097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.602107 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:19.602146 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.602148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.602164 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.602241 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:19.602455 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.603542 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.604659 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.605385 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.605463 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.606137 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.606167 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.606220 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":141,"threads_started":1}
I20260811 10:11:19.606383 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.601473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44066 (local address 127.22.177.3:35111)
0811 10:11:19.601693 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.601698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.601745 (+ 47us) server_negotiation.cc:407] Connection header received
0811 10:11:19.601821 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.601823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.601838 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.601920 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:19.602300 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.603423 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.604721 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.605416 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.605475 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.605831 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.606234 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.606294 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
I20260811 10:11:19.606267 10968 tablet_service.cc:1467] Processing CreateTablet for tablet d9bece753de440c3bed7e3739a1e95d8 (DEFAULT_TABLE table=test-table [id=b96bd15ac3474e4ab11a9f1ad9e3a43b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:19.606688 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9bece753de440c3bed7e3739a1e95d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:19.606987 11274 tablet_service.cc:1467] Processing CreateTablet for tablet d9bece753de440c3bed7e3739a1e95d8 (DEFAULT_TABLE table=test-table [id=b96bd15ac3474e4ab11a9f1ad9e3a43b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:19.607299 11274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9bece753de440c3bed7e3739a1e95d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:19.609380 11373 tablet_bootstrap.cc:492] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: Bootstrap starting.
I20260811 10:11:19.609843 11374 tablet_bootstrap.cc:492] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Bootstrap starting.
I20260811 10:11:19.610170 11373 tablet_bootstrap.cc:654] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:19.610054 11124 tablet_service.cc:1467] Processing CreateTablet for tablet d9bece753de440c3bed7e3739a1e95d8 (DEFAULT_TABLE table=test-table [id=b96bd15ac3474e4ab11a9f1ad9e3a43b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:19.610414 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9bece753de440c3bed7e3739a1e95d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:19.610622 11374 tablet_bootstrap.cc:654] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:19.610805 11373 log.cc:824] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:19.611209 11374 log.cc:824] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:19.611665 11373 tablet_bootstrap.cc:492] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: No bootstrap required, opened a new log
I20260811 10:11:19.611743 11373 ts_tablet_manager.cc:1397] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:11:19.612001 11374 tablet_bootstrap.cc:492] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: No bootstrap required, opened a new log
I20260811 10:11:19.612072 11374 ts_tablet_manager.cc:1397] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:11:19.613584 11377 tablet_bootstrap.cc:492] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Bootstrap starting.
I20260811 10:11:19.613873 11373 raft_consensus.cc:348] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.614075 11373 raft_consensus.cc:374] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:19.614116 11373 raft_consensus.cc:729] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b33690fc65448e9f9d92c460fb70c4, State: Initialized, Role: FOLLOWER
I20260811 10:11:19.614332 11377 tablet_bootstrap.cc:654] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:19.614266 11373 consensus_queue.cc:260] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.614318 11374 raft_consensus.cc:348] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.614485 11374 raft_consensus.cc:374] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:19.614533 11374 raft_consensus.cc:729] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91152224c30844b2842ad94c78564d11, State: Initialized, Role: FOLLOWER
I20260811 10:11:19.614629 11373 ts_tablet_manager.cc:1428] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:19.614683 11374 consensus_queue.cc:260] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.614768 11049 heartbeater.cc:503] Master 127.22.177.62:41799 was elected leader, sending a full tablet report...
I20260811 10:11:19.614939 11377 log.cc:824] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:19.614970 11374 ts_tablet_manager.cc:1428] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:19.615096 11350 heartbeater.cc:503] Master 127.22.177.62:41799 was elected leader, sending a full tablet report...
I20260811 10:11:19.615854 11379 raft_consensus.cc:482] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:19.615962 11379 raft_consensus.cc:504] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.616285 11379 leader_election.cc:288] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24b33690fc65448e9f9d92c460fb70c4 (127.22.177.1:45335), 640316d070d3410d84344490a46e22ac (127.22.177.2:39529)
I20260811 10:11:19.616401 11377 tablet_bootstrap.cc:492] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: No bootstrap required, opened a new log
I20260811 10:11:19.616490 11377 ts_tablet_manager.cc:1397] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 10:11:19.618655 11377 raft_consensus.cc:348] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.618841 11377 raft_consensus.cc:374] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:19.618875 11377 raft_consensus.cc:729] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640316d070d3410d84344490a46e22ac, State: Initialized, Role: FOLLOWER
I20260811 10:11:19.619014 11377 consensus_queue.cc:260] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.619309 11201 heartbeater.cc:503] Master 127.22.177.62:41799 was elected leader, sending a full tablet report...
I20260811 10:11:19.619479 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.616461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:45335 (local address 127.22.177.3:34961)
0811 10:11:19.616518 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.616531 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.616588 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.616862 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.616863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.616864 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:19.616923 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.616925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.617720 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.617721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.618473 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.618475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.619198 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.619200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.619292 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.619296 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.619351 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.619400 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:11:19.619771 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.616684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:34961 (local address 127.22.177.1:45335)
0811 10:11:19.616750 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.616753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.616763 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:19.616787 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.616788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.616799 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.616859 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:19.617002 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.617653 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.618567 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.619055 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.619153 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.619163 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.619666 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.619712 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:11:19.619299 11377 ts_tablet_manager.cc:1428] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 10:11:19.620111 10992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bece753de440c3bed7e3739a1e95d8" candidate_uuid: "91152224c30844b2842ad94c78564d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24b33690fc65448e9f9d92c460fb70c4" is_pre_election: true
I20260811 10:11:19.620253 10992 raft_consensus.cc:2454] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91152224c30844b2842ad94c78564d11 in term 0.
I20260811 10:11:19.620352 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.616615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:33179 (local address 127.22.177.2:39529)
0811 10:11:19.616657 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.616659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.616867 (+ 208us) server_negotiation.cc:407] Connection header received
0811 10:11:19.616960 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.616961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.616969 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.616999 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:19.617199 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.617919 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.619299 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.619846 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.619892 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.619900 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.620222 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.620289 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:19.620488 11226 leader_election.cc:302] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24b33690fc65448e9f9d92c460fb70c4, 91152224c30844b2842ad94c78564d11; no voters:
I20260811 10:11:19.620592 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.616672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:39529 (local address 127.22.177.3:33179)
0811 10:11:19.616805 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.616816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.616892 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.617060 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.617061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.617062 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:19.617127 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.617129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.618023 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.618024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.619180 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.619183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.620057 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.620058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.620137 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.620141 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.620442 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.620488 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260811 10:11:19.620635 11379 raft_consensus.cc:2781] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:11:19.620682 11379 raft_consensus.cc:482] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:11:19.620719 11379 raft_consensus.cc:3037] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:19.620868 11148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bece753de440c3bed7e3739a1e95d8" candidate_uuid: "91152224c30844b2842ad94c78564d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640316d070d3410d84344490a46e22ac" is_pre_election: true
I20260811 10:11:19.620990 11148 raft_consensus.cc:2454] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91152224c30844b2842ad94c78564d11 in term 0.
I20260811 10:11:19.621522 11379 raft_consensus.cc:504] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.621704 11379 leader_election.cc:288] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [CANDIDATE]: Term 1 election: Requested vote from peers 24b33690fc65448e9f9d92c460fb70c4 (127.22.177.1:45335), 640316d070d3410d84344490a46e22ac (127.22.177.2:39529)
I20260811 10:11:19.621869 11148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bece753de440c3bed7e3739a1e95d8" candidate_uuid: "91152224c30844b2842ad94c78564d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640316d070d3410d84344490a46e22ac"
I20260811 10:11:19.621904 10992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bece753de440c3bed7e3739a1e95d8" candidate_uuid: "91152224c30844b2842ad94c78564d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24b33690fc65448e9f9d92c460fb70c4"
I20260811 10:11:19.621966 11148 raft_consensus.cc:3037] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:19.621999 10992 raft_consensus.cc:3037] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:19.622695 10992 raft_consensus.cc:2454] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91152224c30844b2842ad94c78564d11 in term 1.
I20260811 10:11:19.622833 11148 raft_consensus.cc:2454] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91152224c30844b2842ad94c78564d11 in term 1.
I20260811 10:11:19.622856 11226 leader_election.cc:302] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24b33690fc65448e9f9d92c460fb70c4, 91152224c30844b2842ad94c78564d11; no voters:
I20260811 10:11:19.622936 11379 raft_consensus.cc:2781] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:19.623099 11379 raft_consensus.cc:686] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 1 LEADER]: Becoming Leader. State: Replica: 91152224c30844b2842ad94c78564d11, State: Running, Role: LEADER
I20260811 10:11:19.623189 11379 consensus_queue.cc:237] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } }
I20260811 10:11:19.623941 10830 catalog_manager.cc:5495] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91152224c30844b2842ad94c78564d11 (127.22.177.3). New cstate: current_term: 1 leader_uuid: "91152224c30844b2842ad94c78564d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91152224c30844b2842ad94c78564d11" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 35111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 } health_report { overall_health: UNKNOWN } } }
I20260811 10:11:19.634128 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.631687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.22.177.3:35111)
0811 10:11:19.631766 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.631768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.631775 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:11:19.631806 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.631807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.631823 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.631902 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:19.632057 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.632519 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.633310 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.633511 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.633594 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.633780 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:19.633873 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:19.633983 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.634022 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.634033 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 10:11:19.636128 10992 raft_consensus.cc:1261] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 1 FOLLOWER]: Refusing update from remote peer 91152224c30844b2842ad94c78564d11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:19.636138 11148 raft_consensus.cc:1261] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 1 FOLLOWER]: Refusing update from remote peer 91152224c30844b2842ad94c78564d11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:19.636557 11383 consensus_queue.cc:1035] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "640316d070d3410d84344490a46e22ac" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 39529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:19.636705 11379 consensus_queue.cc:1035] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24b33690fc65448e9f9d92c460fb70c4" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 45335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:19.639459 11387 mvcc.cc:205] Tried to move back new op lower bound from 7317270854186659840 to 7317270854137245696. Current Snapshot: MvccSnapshot[applied={T|T < 7317270854186659840}]
I20260811 10:11:19.640573 11388 mvcc.cc:205] Tried to move back new op lower bound from 7317270854186659840 to 7317270854137245696. Current Snapshot: MvccSnapshot[applied={T|T < 7317270854186659840}]
I20260811 10:11:19.642347 10830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:19.642467 10830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:19.643306 10830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b575e663d235410c9c227a889885566a: Sending DeleteTablet for 3 replicas of tablet d9bece753de440c3bed7e3739a1e95d8
I20260811 10:11:19.643689 11274 tablet_service.cc:1514] Processing DeleteTablet for tablet d9bece753de440c3bed7e3739a1e95d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:19 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:44066
I20260811 10:11:19.643703 10968 tablet_service.cc:1514] Processing DeleteTablet for tablet d9bece753de440c3bed7e3739a1e95d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:19 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:42428
I20260811 10:11:19.643718 11124 tablet_service.cc:1514] Processing DeleteTablet for tablet d9bece753de440c3bed7e3739a1e95d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:19 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:50818
I20260811 10:11:19.643929 11398 tablet_replica.cc:330] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: stopping tablet replica
I20260811 10:11:19.644025 11398 raft_consensus.cc:2229] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:11:19.644137 11398 raft_consensus.cc:2258] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:19.644351 11399 tablet_replica.cc:330] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4: stopping tablet replica
I20260811 10:11:19.644418 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 10899
I20260811 10:11:19.644469 11399 raft_consensus.cc:2229] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:19.644586 11399 raft_consensus.cc:2258] T d9bece753de440c3bed7e3739a1e95d8 P 24b33690fc65448e9f9d92c460fb70c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:19.645977 11398 ts_tablet_manager.cc:1905] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:19.648142 11400 tablet_replica.cc:330] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: stopping tablet replica
I20260811 10:11:19.648203 11398 ts_tablet_manager.cc:1918] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:19.648283 11398 log.cc:1196] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/d9bece753de440c3bed7e3739a1e95d8
I20260811 10:11:19.648461 11400 raft_consensus.cc:2229] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:19.648578 11398 ts_tablet_manager.cc:1939] T d9bece753de440c3bed7e3739a1e95d8 P 91152224c30844b2842ad94c78564d11: Deleting consensus metadata
I20260811 10:11:19.648723 11400 raft_consensus.cc:2258] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:19.649046 10812 catalog_manager.cc:4841] TS 91152224c30844b2842ad94c78564d11 (127.22.177.3:35111): tablet d9bece753de440c3bed7e3739a1e95d8 (table test-table [id=b96bd15ac3474e4ab11a9f1ad9e3a43b]) successfully deleted
I20260811 10:11:19.649365 11400 ts_tablet_manager.cc:1905] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:19.650985 11400 ts_tablet_manager.cc:1918] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:19.651094 11400 log.cc:1196] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/d9bece753de440c3bed7e3739a1e95d8
I20260811 10:11:19.651378 11400 ts_tablet_manager.cc:1939] T d9bece753de440c3bed7e3739a1e95d8 P 640316d070d3410d84344490a46e22ac: Deleting consensus metadata
I20260811 10:11:19.651870 10812 catalog_manager.cc:4841] TS 640316d070d3410d84344490a46e22ac (127.22.177.2:39529): tablet d9bece753de440c3bed7e3739a1e95d8 (table test-table [id=b96bd15ac3474e4ab11a9f1ad9e3a43b]) successfully deleted
W20260811 10:11:19.653640 10811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:45335 (error 108)
I20260811 10:11:19.654268 11365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.654117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:45335 (local address 127.0.0.1:42438)
0811 10:11:19.654186 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.654229 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:45335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:11:19.654325 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 11055
W20260811 10:11:19.655707 10811 catalog_manager.cc:4584] TS 24b33690fc65448e9f9d92c460fb70c4 (127.22.177.1:45335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9bece753de440c3bed7e3739a1e95d8: Network error: Client connection negotiation failed: client connection to 127.22.177.1:45335: connect: Connection refused (error 111)
I20260811 10:11:19.663224 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 11207
I20260811 10:11:19.670250 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 10789
2026-08-11T10:11:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2073 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T10:11:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:19Z Disabled control of system clock
I20260811 10:11:19.704216 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:38611
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:38611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:19.787530 11407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:19.787755 11407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:19.787778 11407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:19.788964 11407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:19.789004 11407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:19.789017 11407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:19.789031 11407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:19.789043 11407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:19.790503 11407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:38611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:38611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:19.790767 11407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:19.791076 11407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:19.796211 11413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.796247 11414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.796337 11416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.796615 11407 server_base.cc:1034] running on GCE node
I20260811 10:11:19.796849 11407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:19.797101 11407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:19.798261 11407 hybrid_clock.cc:648] HybridClock initialized: now 1786443079798227 us; error 53 us; skew 500 ppm
I20260811 10:11:19.799748 11407 webserver.cc:460] Webserver started at http://127.22.177.62:42259/ using document root <none> and password file <none>
I20260811 10:11:19.800038 11407 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:19.800093 11407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:19.800348 11407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:19.801584 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "af1690c3e44242c8a39a38f87f394367"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "f09b8c7b6dfb4c287a6b256e68a90efa"
server_key_iv: "9213e6a9bda814b694cf3812faa9e375"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.802085 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "af1690c3e44242c8a39a38f87f394367"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "f09b8c7b6dfb4c287a6b256e68a90efa"
server_key_iv: "9213e6a9bda814b694cf3812faa9e375"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.803633 11407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:19.804606 11423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.804893 11407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:19.805042 11407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "af1690c3e44242c8a39a38f87f394367"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "f09b8c7b6dfb4c287a6b256e68a90efa"
server_key_iv: "9213e6a9bda814b694cf3812faa9e375"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.805131 11407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:19.813395 11407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:19.813787 11407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:19.813948 11407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:19.818416 11407 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:38611
I20260811 10:11:19.818451 11484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:38611 every 8 connection(s)
I20260811 10:11:19.818828 11407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:19.819252 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 11407
I20260811 10:11:19.819377 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:19.819612 23236 external_mini_cluster.cc:1428] Setting key dab1a65147d1660250410f44428324d0
I20260811 10:11:19.820160 11485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:19.823606 11485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367: Bootstrap starting.
I20260811 10:11:19.824306 11485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:19.824610 11485 log.cc:824] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:19.825357 11485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367: No bootstrap required, opened a new log
I20260811 10:11:19.826303 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.820264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.22.177.62:38611)
0811 10:11:19.820508 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.820516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.820546 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:11:19.821336 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.821345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.821417 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.821553 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.821914 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.822979 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.824137 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.824673 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.824814 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.825613 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.825618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.825618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:19.825807 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.825854 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:19.825865 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.825950 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.826028 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.826116 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":130,"threads_started":1}
I20260811 10:11:19.827626 11485 raft_consensus.cc:348] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af1690c3e44242c8a39a38f87f394367" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38611 } }
I20260811 10:11:19.827796 11485 raft_consensus.cc:374] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:19.827847 11485 raft_consensus.cc:729] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af1690c3e44242c8a39a38f87f394367, State: Initialized, Role: FOLLOWER
I20260811 10:11:19.827981 11485 consensus_queue.cc:260] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af1690c3e44242c8a39a38f87f394367" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38611 } }
I20260811 10:11:19.828064 11485 raft_consensus.cc:388] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:19.828109 11485 raft_consensus.cc:482] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:19.828161 11485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:19.828742 11485 raft_consensus.cc:504] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af1690c3e44242c8a39a38f87f394367" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38611 } }
I20260811 10:11:19.828867 11485 leader_election.cc:302] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af1690c3e44242c8a39a38f87f394367; no voters:
I20260811 10:11:19.829075 11485 leader_election.cc:288] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:19.829126 11490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:19.829380 11485 sys_catalog.cc:564] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:19.829417 11490 raft_consensus.cc:686] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [term 1 LEADER]: Becoming Leader. State: Replica: af1690c3e44242c8a39a38f87f394367, State: Running, Role: LEADER
I20260811 10:11:19.829515 11490 consensus_queue.cc:237] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af1690c3e44242c8a39a38f87f394367" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38611 } }
I20260811 10:11:19.829990 11492 sys_catalog.cc:455] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af1690c3e44242c8a39a38f87f394367. Latest consensus state: current_term: 1 leader_uuid: "af1690c3e44242c8a39a38f87f394367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af1690c3e44242c8a39a38f87f394367" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38611 } } }
I20260811 10:11:19.829979 11491 sys_catalog.cc:455] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af1690c3e44242c8a39a38f87f394367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af1690c3e44242c8a39a38f87f394367" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38611 } } }
I20260811 10:11:19.830094 11492 sys_catalog.cc:458] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:19.830099 11491 sys_catalog.cc:458] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:19.830437 11500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:19.830977 11500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:19.832538 11500 catalog_manager.cc:1347] Generated new cluster ID: 146a972c24734ca8ac5c9c3cb2b1c56b
I20260811 10:11:19.832597 11500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:19.845363 11500 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:19.845975 11500 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:19.850008 11500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367: Generated new TSK 0
I20260811 10:11:19.850229 11500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:19.855167 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38611
--builtin_ntp_servers=127.22.177.20:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:19.950520 11516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:19.950762 11516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:19.950793 11516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:19.952243 11516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:19.952298 11516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:19.952327 11516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:11:19.953874 11516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:19.954154 11516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:19.954499 11516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:19.960067 11521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.960067 11522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:19.960107 11524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:19.960644 11516 server_base.cc:1034] running on GCE node
I20260811 10:11:19.960872 11516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:19.961169 11516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:19.962347 11516 hybrid_clock.cc:648] HybridClock initialized: now 1786443079962325 us; error 38 us; skew 500 ppm
I20260811 10:11:19.963692 11516 webserver.cc:460] Webserver started at http://127.22.177.1:43467/ using document root <none> and password file <none>
I20260811 10:11:19.963980 11516 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:19.964035 11516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:19.964170 11516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:19.965293 11516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "1fa97cd2ec4d440989dee4a82dc21f86"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "fb7bf5297960885cf14ccf9aa47aee5c"
server_key_iv: "42afe290ece12c05a8d4c0e589acd7b0"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.965704 11516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "1fa97cd2ec4d440989dee4a82dc21f86"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "fb7bf5297960885cf14ccf9aa47aee5c"
server_key_iv: "42afe290ece12c05a8d4c0e589acd7b0"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.967100 11516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:19.967947 11532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.968182 11516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:19.968312 11516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "1fa97cd2ec4d440989dee4a82dc21f86"
format_stamp: "Formatted at 2026-08-11 10:11:19 on dist-test-slave-g350"
server_key: "fb7bf5297960885cf14ccf9aa47aee5c"
server_key_iv: "42afe290ece12c05a8d4c0e589acd7b0"
server_key_version: "encryptionkey@0"
I20260811 10:11:19.968390 11516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:19.984112 11516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:19.984472 11516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:19.984639 11516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:19.984899 11516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:19.985266 11516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:19.985328 11516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.985370 11516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:19.985414 11516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:19.991820 11516 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:41229
I20260811 10:11:19.991883 11664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:41229 every 8 connection(s)
I20260811 10:11:19.992272 11516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:11:19.997728 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.993076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:52319 (local address 127.22.177.62:38611)
0811 10:11:19.993139 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:19.993141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:19.993377 (+ 236us) server_negotiation.cc:407] Connection header received
0811 10:11:19.993974 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:19.993975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:19.993990 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:19.994045 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:19.994774 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.995271 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.996294 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.996529 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.996586 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.997466 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:19.997467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:19.997467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:19.997471 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:19.997481 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:19.997484 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:19.997520 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:19.997583 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 10:11:19.997669 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:11:19.998020 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:19.993073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:38611 (local address 127.22.177.1:52319)
0811 10:11:19.993250 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:19.993296 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:19.993882 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:19.994109 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:19.994112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:19.994120 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:19.994691 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.994696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.995385 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.995386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.996213 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:19.996215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:19.996608 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:19.996609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:19.996715 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:19.997325 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:19.997326 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:19.997368 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:19.997392 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:19.997522 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:19.997524 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:19.997527 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:19.997573 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 10:11:19.997705 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260811 10:11:19.998620 11665 heartbeater.cc:347] Connected to a master server at 127.22.177.62:38611
I20260811 10:11:19.998728 11665 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:19.998919 11665 heartbeater.cc:511] Master 127.22.177.62:38611 requested a full tablet report, sending...
I20260811 10:11:19.999462 11444 ts_manager.cc:194] Registered new tserver with Master: 1fa97cd2ec4d440989dee4a82dc21f86 (127.22.177.1:41229)
I20260811 10:11:20.000069 11444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.1:52319
I20260811 10:11:20.002000 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 11516
I20260811 10:11:20.002115 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:11:20.002393 23236 external_mini_cluster.cc:1428] Setting key d151df03534aa276db66e5b08e50c476
I20260811 10:11:20.003456 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38611
--builtin_ntp_servers=127.22.177.20:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:20.089821 11668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:20.090090 11668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:20.090127 11668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:20.091324 11668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:20.091401 11668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:20.091444 11668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:11:20.092867 11668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:20.093168 11668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:20.093498 11668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:20.099247 11673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:20.099265 11677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:20.099243 11674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:20.099668 11668 server_base.cc:1034] running on GCE node
I20260811 10:11:20.099900 11668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:20.100183 11668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:20.101404 11668 hybrid_clock.cc:648] HybridClock initialized: now 1786443080101406 us; error 62 us; skew 500 ppm
I20260811 10:11:20.103050 11668 webserver.cc:460] Webserver started at http://127.22.177.2:44535/ using document root <none> and password file <none>
I20260811 10:11:20.103359 11668 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:20.103438 11668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:20.103600 11668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:20.104730 11668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "a2d7caf3f69d45c186f748dff511d799"
format_stamp: "Formatted at 2026-08-11 10:11:20 on dist-test-slave-g350"
server_key: "67938cf088b8a69ddabced79a28d0c2c"
server_key_iv: "8ffe14f535411df3a0513c5b932a27cc"
server_key_version: "encryptionkey@0"
I20260811 10:11:20.105132 11668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "a2d7caf3f69d45c186f748dff511d799"
format_stamp: "Formatted at 2026-08-11 10:11:20 on dist-test-slave-g350"
server_key: "67938cf088b8a69ddabced79a28d0c2c"
server_key_iv: "8ffe14f535411df3a0513c5b932a27cc"
server_key_version: "encryptionkey@0"
I20260811 10:11:20.106729 11668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:20.107743 11685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:20.108078 11668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:20.108196 11668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "a2d7caf3f69d45c186f748dff511d799"
format_stamp: "Formatted at 2026-08-11 10:11:20 on dist-test-slave-g350"
server_key: "67938cf088b8a69ddabced79a28d0c2c"
server_key_iv: "8ffe14f535411df3a0513c5b932a27cc"
server_key_version: "encryptionkey@0"
I20260811 10:11:20.108282 11668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:20.126098 11668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:20.126478 11668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:20.126633 11668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:20.126926 11668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:20.127310 11668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:20.127374 11668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:20.127411 11668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:20.127439 11668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:20.134187 11668 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:43997
I20260811 10:11:20.134236 11814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:43997 every 8 connection(s)
I20260811 10:11:20.134640 11668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:11:20.140341 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 11668
I20260811 10:11:20.140532 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:11:20.140789 23236 external_mini_cluster.cc:1428] Setting key 4db9a6daa2928cb7f096c75388a72606
I20260811 10:11:20.141285 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.135540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:58409 (local address 127.22.177.62:38611)
0811 10:11:20.135628 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.135630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.135907 (+ 277us) server_negotiation.cc:407] Connection header received
0811 10:11:20.136563 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.136564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.136575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.136644 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:20.137515 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.138116 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.139270 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.139491 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.139552 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.140917 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:20.140919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:20.140919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:20.140925 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:20.140945 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:20.140949 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:20.141037 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.141167 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.141201 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 10:11:20.141763 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.135504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:38611 (local address 127.22.177.2:58409)
0811 10:11:20.135752 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.135792 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.136425 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.136715 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.136719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.136728 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:20.137390 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.137396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.138354 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.138355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.139172 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.139175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.139740 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.139741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.139875 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.140613 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:20.140613 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:20.140706 (+ 93us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:20.140742 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:20.141054 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:20.141063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:20.141067 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.141170 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.141284 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
I20260811 10:11:20.142194 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38611
--builtin_ntp_servers=127.22.177.20:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:11:20.142233 11815 heartbeater.cc:347] Connected to a master server at 127.22.177.62:38611
I20260811 10:11:20.142338 11815 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:20.142524 11815 heartbeater.cc:511] Master 127.22.177.62:38611 requested a full tablet report, sending...
I20260811 10:11:20.142963 11444 ts_manager.cc:194] Registered new tserver with Master: a2d7caf3f69d45c186f748dff511d799 (127.22.177.2:43997)
I20260811 10:11:20.143373 11444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.2:58409
W20260811 10:11:20.228983 11818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:20.229336 11818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:20.229413 11818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:20.230949 11818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:20.231014 11818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:20.231060 11818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:11:20.232513 11818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:20.232797 11818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:20.233134 11818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:20.238652 11826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:20.238665 11828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:20.238669 11824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:20.239035 11818 server_base.cc:1034] running on GCE node
I20260811 10:11:20.239277 11818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:20.239538 11818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:20.240741 11818 hybrid_clock.cc:648] HybridClock initialized: now 1786443080240699 us; error 44 us; skew 500 ppm
I20260811 10:11:20.242264 11818 webserver.cc:460] Webserver started at http://127.22.177.3:39127/ using document root <none> and password file <none>
I20260811 10:11:20.242522 11818 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:20.242594 11818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:20.242753 11818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:20.243836 11818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "472fc4ca7f85422ab1e08ccf3fa497c7"
format_stamp: "Formatted at 2026-08-11 10:11:20 on dist-test-slave-g350"
server_key: "35f275edd22d48bc4df34264e1d680bc"
server_key_iv: "3d36cb667f04d20f30fc0c3a8c88a234"
server_key_version: "encryptionkey@0"
I20260811 10:11:20.244217 11818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "472fc4ca7f85422ab1e08ccf3fa497c7"
format_stamp: "Formatted at 2026-08-11 10:11:20 on dist-test-slave-g350"
server_key: "35f275edd22d48bc4df34264e1d680bc"
server_key_iv: "3d36cb667f04d20f30fc0c3a8c88a234"
server_key_version: "encryptionkey@0"
I20260811 10:11:20.245574 11818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:11:20.246590 11834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:20.246882 11818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:20.247016 11818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "472fc4ca7f85422ab1e08ccf3fa497c7"
format_stamp: "Formatted at 2026-08-11 10:11:20 on dist-test-slave-g350"
server_key: "35f275edd22d48bc4df34264e1d680bc"
server_key_iv: "3d36cb667f04d20f30fc0c3a8c88a234"
server_key_version: "encryptionkey@0"
I20260811 10:11:20.247115 11818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:20.255551 11818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:20.255935 11818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:20.256107 11818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:20.256397 11818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:20.256778 11818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:20.256839 11818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:20.256886 11818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:20.256914 11818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:20.263811 11818 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:39899
I20260811 10:11:20.263877 11960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:39899 every 8 connection(s)
I20260811 10:11:20.264281 11818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:11:20.268671 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 11818
I20260811 10:11:20.268826 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:11:20.269047 23236 external_mini_cluster.cc:1428] Setting key 1fd85fc7f807629667d9684ecbfcaa96
I20260811 10:11:20.270418 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.265083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:60829 (local address 127.22.177.62:38611)
0811 10:11:20.265170 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.265172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.265435 (+ 263us) server_negotiation.cc:407] Connection header received
0811 10:11:20.266022 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.266024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.266039 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.266105 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:20.266858 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.267392 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.268449 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.268691 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.268791 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.270068 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:20.270069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:20.270069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:20.270073 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:20.270113 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:20.270118 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:20.270178 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.270315 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.270346 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 10:11:20.270718 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.265073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:38611 (local address 127.22.177.3:60829)
0811 10:11:20.265297 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.265333 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.265898 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.266189 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.266192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.266200 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:20.266745 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.266750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.267512 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.267514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.268339 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.268341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.268808 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.268809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.268995 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.269894 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:20.269894 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:20.269952 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:20.269983 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:20.270196 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:20.270204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:20.270207 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.270299 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.270421 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260811 10:11:20.271263 11961 heartbeater.cc:347] Connected to a master server at 127.22.177.62:38611
I20260811 10:11:20.271382 11961 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:20.271631 11961 heartbeater.cc:511] Master 127.22.177.62:38611 requested a full tablet report, sending...
I20260811 10:11:20.272078 11444 ts_manager.cc:194] Registered new tserver with Master: 472fc4ca7f85422ab1e08ccf3fa497c7 (127.22.177.3:39899)
I20260811 10:11:20.272435 11444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.3:60829
I20260811 10:11:20.281518 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:11:20.286499 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.283095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.22.177.62:38611)
0811 10:11:20.283166 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.283168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.283228 (+ 60us) server_negotiation.cc:407] Connection header received
0811 10:11:20.283383 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.283384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.283395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.283455 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:20.283680 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.284171 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.285053 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.285299 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.285364 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.286247 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:20.286248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:20.286248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:20.286253 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:20.286268 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:20.286271 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:20.286358 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.286429 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.286439 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:11:20.288264 11444 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:11:20.289927 11444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:11:20.296869 11973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.292696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.22.177.2:43997)
0811 10:11:20.292907 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.292912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.293082 (+ 170us) server_negotiation.cc:407] Connection header received
0811 10:11:20.293126 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.293127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.293143 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.293208 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:20.293445 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.294311 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.295805 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.296459 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.296550 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.296654 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.296739 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.296779 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260811 10:11:20.296857 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.292461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:43997 (local address 127.0.0.1:52782)
0811 10:11:20.292876 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.292920 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.293000 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.293287 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.293290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.293293 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:20.293363 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.293366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.294406 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.294407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.295719 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.295722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.296552 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.296553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.296634 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.296648 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.296725 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.296779 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":104,"threads_started":1}
I20260811 10:11:20.297114 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.292493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:41229 (local address 127.0.0.1:41804)
0811 10:11:20.292911 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.292924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.293000 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.293412 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.293413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.293415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:20.293456 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.293457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.294803 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.294804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.295863 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.295864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.296876 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.296877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.296945 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.296949 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.296995 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.297038 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":87,"threads_started":1}
I20260811 10:11:20.297291 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.292752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:39899 (local address 127.0.0.1:35604)
0811 10:11:20.293280 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.293290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.293345 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.293660 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.293661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.293663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:20.293700 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.293701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.294890 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.294891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.296062 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.296065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.297045 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.297046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.297102 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.297105 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.297177 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.297217 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":93,"threads_started":1}
I20260811 10:11:20.297437 11972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.292639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.22.177.1:41229)
0811 10:11:20.293001 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.293005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.293016 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:11:20.293071 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.293073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.293088 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.293169 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:20.293545 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.294712 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.295955 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.296658 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.296747 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.296834 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.297289 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.297336 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":122,"threads_started":1}
I20260811 10:11:20.297574 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.292870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.22.177.3:39899)
0811 10:11:20.293201 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.293206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.293340 (+ 134us) server_negotiation.cc:407] Connection header received
0811 10:11:20.293437 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.293439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.293454 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.293560 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:20.293784 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.294826 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.296161 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.296869 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.296933 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.297013 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.297436 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.297483 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":86,"threads_started":1}
I20260811 10:11:20.298079 11740 tablet_service.cc:1467] Processing CreateTablet for tablet 7b5af7fabca74015b9b16637d0e0c253 (DEFAULT_TABLE table=test-table [id=2d566096b8404014bcac5ec9864ade4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:20.298121 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 7b5af7fabca74015b9b16637d0e0c253 (DEFAULT_TABLE table=test-table [id=2d566096b8404014bcac5ec9864ade4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:20.298480 11740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b5af7fabca74015b9b16637d0e0c253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:20.298489 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b5af7fabca74015b9b16637d0e0c253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:20.301172 11978 tablet_bootstrap.cc:492] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: Bootstrap starting.
I20260811 10:11:20.301169 11977 tablet_bootstrap.cc:492] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Bootstrap starting.
I20260811 10:11:20.301724 11885 tablet_service.cc:1467] Processing CreateTablet for tablet 7b5af7fabca74015b9b16637d0e0c253 (DEFAULT_TABLE table=test-table [id=2d566096b8404014bcac5ec9864ade4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:20.301898 11978 tablet_bootstrap.cc:654] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:20.302042 11977 tablet_bootstrap.cc:654] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:20.302112 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b5af7fabca74015b9b16637d0e0c253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:20.302253 11978 log.cc:824] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:20.302460 11977 log.cc:824] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:20.303158 11978 tablet_bootstrap.cc:492] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: No bootstrap required, opened a new log
I20260811 10:11:20.303184 11977 tablet_bootstrap.cc:492] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: No bootstrap required, opened a new log
I20260811 10:11:20.303233 11978 ts_tablet_manager.cc:1397] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:11:20.303277 11977 ts_tablet_manager.cc:1397] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:11:20.305276 11977 raft_consensus.cc:348] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.305442 11977 raft_consensus.cc:374] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:20.305487 11977 raft_consensus.cc:729] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7caf3f69d45c186f748dff511d799, State: Initialized, Role: FOLLOWER
I20260811 10:11:20.305647 11977 consensus_queue.cc:260] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.305943 11977 ts_tablet_manager.cc:1428] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 10:11:20.306037 11981 tablet_bootstrap.cc:492] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Bootstrap starting.
I20260811 10:11:20.306069 11815 heartbeater.cc:503] Master 127.22.177.62:38611 was elected leader, sending a full tablet report...
I20260811 10:11:20.305976 11978 raft_consensus.cc:348] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.306146 11978 raft_consensus.cc:374] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:20.306197 11978 raft_consensus.cc:729] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa97cd2ec4d440989dee4a82dc21f86, State: Initialized, Role: FOLLOWER
I20260811 10:11:20.306322 11978 consensus_queue.cc:260] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.306592 11978 ts_tablet_manager.cc:1428] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 10:11:20.306654 11665 heartbeater.cc:503] Master 127.22.177.62:38611 was elected leader, sending a full tablet report...
I20260811 10:11:20.306721 11981 tablet_bootstrap.cc:654] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:20.307081 11981 log.cc:824] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:20.307888 11981 tablet_bootstrap.cc:492] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: No bootstrap required, opened a new log
I20260811 10:11:20.307945 11981 ts_tablet_manager.cc:1397] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:11:20.310055 11981 raft_consensus.cc:348] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.310189 11981 raft_consensus.cc:374] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:20.310210 11981 raft_consensus.cc:729] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 472fc4ca7f85422ab1e08ccf3fa497c7, State: Initialized, Role: FOLLOWER
I20260811 10:11:20.310350 11981 consensus_queue.cc:260] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.310606 11981 ts_tablet_manager.cc:1428] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:11:20.310683 11961 heartbeater.cc:503] Master 127.22.177.62:38611 was elected leader, sending a full tablet report...
I20260811 10:11:20.366997 11982 raft_consensus.cc:482] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:20.367147 11982 raft_consensus.cc:504] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.367741 11982 leader_election.cc:288] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 472fc4ca7f85422ab1e08ccf3fa497c7 (127.22.177.3:39899), 1fa97cd2ec4d440989dee4a82dc21f86 (127.22.177.1:41229)
I20260811 10:11:20.370924 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.367939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:39899 (local address 127.22.177.2:51831)
0811 10:11:20.368006 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.368020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.368068 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.368399 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.368400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.368401 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:20.368514 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.368515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.369178 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.369180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.370062 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.370065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.370616 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.370617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.370686 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.370689 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.370793 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.370845 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":24}
I20260811 10:11:20.370975 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.368148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:51831 (local address 127.22.177.3:39899)
0811 10:11:20.368202 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.368205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.368213 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:20.368299 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.368301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.368313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.368395 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:20.368596 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.369088 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.370159 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.370520 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.370616 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.370626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.370821 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.370875 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:11:20.371078 11972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.368133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:45387 (local address 127.22.177.1:41229)
0811 10:11:20.368197 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.368200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.368213 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:11:20.368314 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.368315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.368330 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.368390 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:20.368591 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.369108 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.370227 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.370635 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.370730 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.370739 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.370943 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.370997 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:11:20.371089 11989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.367966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:41229 (local address 127.22.177.2:45387)
0811 10:11:20.368158 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.368170 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:20.368230 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:20.368396 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:20.368397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:20.368398 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:20.368500 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.368502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.369208 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.369209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.370116 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:20.370119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.370731 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.370732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:20.370857 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.370861 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:20.370925 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 10:11:20.370973 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260811 10:11:20.371354 11908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5af7fabca74015b9b16637d0e0c253" candidate_uuid: "a2d7caf3f69d45c186f748dff511d799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" is_pre_election: true
I20260811 10:11:20.371444 11614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5af7fabca74015b9b16637d0e0c253" candidate_uuid: "a2d7caf3f69d45c186f748dff511d799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" is_pre_election: true
I20260811 10:11:20.371526 11908 raft_consensus.cc:2454] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2d7caf3f69d45c186f748dff511d799 in term 0.
I20260811 10:11:20.371639 11614 raft_consensus.cc:2454] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2d7caf3f69d45c186f748dff511d799 in term 0.
I20260811 10:11:20.371825 11689 leader_election.cc:302] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 472fc4ca7f85422ab1e08ccf3fa497c7, a2d7caf3f69d45c186f748dff511d799; no voters:
I20260811 10:11:20.372025 11982 raft_consensus.cc:2781] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:11:20.372090 11982 raft_consensus.cc:482] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:11:20.372145 11982 raft_consensus.cc:3037] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:20.372905 11982 raft_consensus.cc:504] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.373073 11982 leader_election.cc:288] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [CANDIDATE]: Term 1 election: Requested vote from peers 472fc4ca7f85422ab1e08ccf3fa497c7 (127.22.177.3:39899), 1fa97cd2ec4d440989dee4a82dc21f86 (127.22.177.1:41229)
I20260811 10:11:20.373248 11908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5af7fabca74015b9b16637d0e0c253" candidate_uuid: "a2d7caf3f69d45c186f748dff511d799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7"
I20260811 10:11:20.373327 11908 raft_consensus.cc:3037] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:20.373306 11614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5af7fabca74015b9b16637d0e0c253" candidate_uuid: "a2d7caf3f69d45c186f748dff511d799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa97cd2ec4d440989dee4a82dc21f86"
I20260811 10:11:20.373406 11614 raft_consensus.cc:3037] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:20.374261 11908 raft_consensus.cc:2454] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2d7caf3f69d45c186f748dff511d799 in term 1.
I20260811 10:11:20.374353 11614 raft_consensus.cc:2454] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2d7caf3f69d45c186f748dff511d799 in term 1.
I20260811 10:11:20.374457 11689 leader_election.cc:302] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 472fc4ca7f85422ab1e08ccf3fa497c7, a2d7caf3f69d45c186f748dff511d799; no voters:
I20260811 10:11:20.374559 11982 raft_consensus.cc:2781] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:20.374732 11982 raft_consensus.cc:686] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 1 LEADER]: Becoming Leader. State: Replica: a2d7caf3f69d45c186f748dff511d799, State: Running, Role: LEADER
I20260811 10:11:20.374876 11982 consensus_queue.cc:237] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } }
I20260811 10:11:20.375855 11442 catalog_manager.cc:5495] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7caf3f69d45c186f748dff511d799 (127.22.177.2). New cstate: current_term: 1 leader_uuid: "a2d7caf3f69d45c186f748dff511d799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2d7caf3f69d45c186f748dff511d799" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 43997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 } health_report { overall_health: UNKNOWN } } }
W20260811 10:11:20.385293 11816 tablet.cc:2369] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:11:20.392221 11973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.389631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.22.177.2:43997)
0811 10:11:20.389682 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:20.389684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:20.389803 (+ 119us) server_negotiation.cc:407] Connection header received
0811 10:11:20.389907 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:20.389908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:20.389928 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:20.389992 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:20.390144 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.390621 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.391475 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:20.391669 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:20.391760 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:20.391904 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:20.391991 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:20.392063 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:20.392124 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 10:11:20.392135 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:20.394299 11908 raft_consensus.cc:1261] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 1 FOLLOWER]: Refusing update from remote peer a2d7caf3f69d45c186f748dff511d799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:20.394348 11614 raft_consensus.cc:1261] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86 [term 1 FOLLOWER]: Refusing update from remote peer a2d7caf3f69d45c186f748dff511d799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:20.394722 11982 consensus_queue.cc:1035] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "472fc4ca7f85422ab1e08ccf3fa497c7" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:20.394925 11990 consensus_queue.cc:1035] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa97cd2ec4d440989dee4a82dc21f86" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 41229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:20.397871 11995 mvcc.cc:205] Tried to move back new op lower bound from 7317270857291878400 to 7317270857216471040. Current Snapshot: MvccSnapshot[applied={T|T < 7317270857291878400}]
I20260811 10:11:20.400401 11444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:20.400520 11444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:20.401404 11444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af1690c3e44242c8a39a38f87f394367: Sending DeleteTablet for 3 replicas of tablet 7b5af7fabca74015b9b16637d0e0c253
I20260811 10:11:20.401854 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b5af7fabca74015b9b16637d0e0c253 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:20 UTC) from {username='slave'} at 127.0.0.1:41804
I20260811 10:11:20.402117 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 11516
I20260811 10:11:20.402113 11740 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b5af7fabca74015b9b16637d0e0c253 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:20 UTC) from {username='slave'} at 127.0.0.1:52782
I20260811 10:11:20.402148 12003 tablet_replica.cc:330] T 7b5af7fabca74015b9b16637d0e0c253 P 1fa97cd2ec4d440989dee4a82dc21f86: stopping tablet replica
I20260811 10:11:20.404666 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b5af7fabca74015b9b16637d0e0c253 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:20 UTC) from {username='slave'} at 127.0.0.1:35604
I20260811 10:11:20.406165 12004 tablet_replica.cc:330] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: stopping tablet replica
I20260811 10:11:20.406322 12004 raft_consensus.cc:2229] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:11:20.406490 12004 raft_consensus.cc:2258] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:20.406988 12004 ts_tablet_manager.cc:1905] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:20.408968 12004 ts_tablet_manager.cc:1918] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:20.409070 12004 log.cc:1196] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/7b5af7fabca74015b9b16637d0e0c253
I20260811 10:11:20.409355 12004 ts_tablet_manager.cc:1939] T 7b5af7fabca74015b9b16637d0e0c253 P a2d7caf3f69d45c186f748dff511d799: Deleting consensus metadata
I20260811 10:11:20.409754 12005 tablet_replica.cc:330] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: stopping tablet replica
I20260811 10:11:20.409967 12005 raft_consensus.cc:2229] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:20.409951 11430 catalog_manager.cc:4841] TS a2d7caf3f69d45c186f748dff511d799 (127.22.177.2:43997): tablet 7b5af7fabca74015b9b16637d0e0c253 (table test-table [id=2d566096b8404014bcac5ec9864ade4e]) successfully deleted
I20260811 10:11:20.410182 12005 raft_consensus.cc:2258] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:20.410724 12005 ts_tablet_manager.cc:1905] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 10:11:20.412276 11427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:41229 (error 108)
I20260811 10:11:20.412629 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 11668
I20260811 10:11:20.416085 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:20.412740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:41229 (local address 127.0.0.1:41818)
0811 10:11:20.415972 (+ 3232us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:20.416049 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3201}
W20260811 10:11:20.418673 11427 catalog_manager.cc:4584] TS 1fa97cd2ec4d440989dee4a82dc21f86 (127.22.177.1:41229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b5af7fabca74015b9b16637d0e0c253: Network error: Client connection negotiation failed: client connection to 127.22.177.1:41229: connect: Connection refused (error 111)
I20260811 10:11:20.422185 12005 ts_tablet_manager.cc:1918] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:20.422417 12005 log.cc:1196] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/7b5af7fabca74015b9b16637d0e0c253
I20260811 10:11:20.422768 12005 ts_tablet_manager.cc:1939] T 7b5af7fabca74015b9b16637d0e0c253 P 472fc4ca7f85422ab1e08ccf3fa497c7: Deleting consensus metadata
I20260811 10:11:20.423620 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 11818
I20260811 10:11:20.423736 11430 catalog_manager.cc:4841] TS 472fc4ca7f85422ab1e08ccf3fa497c7 (127.22.177.3:39899): tablet 7b5af7fabca74015b9b16637d0e0c253 (table test-table [id=2d566096b8404014bcac5ec9864ade4e]) successfully deleted
I20260811 10:11:20.430655 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 11407
2026-08-11T10:11:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (759 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:20 dist-test-slave-g350 krb5kdc[12009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:20 dist-test-slave-g350 krb5kdc[12009](info): set up 2 sockets
Aug 11 10:11:20 dist-test-slave-g350 krb5kdc[12009](info): commencing operation
krb5kdc: starting...
W20260811 10:11:21.630024 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:21 dist-test-slave-g350 krb5kdc[12009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:21.781510 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:38145
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:38145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:21.868283 12024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:21.868525 12024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:21.868563 12024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:21.869771 12024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:21.869845 12024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:21.869869 12024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:21.869890 12024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:21.869918 12024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:21.871399 12024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:38145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:38145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:21.871685 12024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:21.871930 12024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:21.877173 12030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:21.877174 12032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:21.877564 12024 server_base.cc:1034] running on GCE node
W20260811 10:11:21.877207 12029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:21.878345 12024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:21.878672 12024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:21.879984 12024 hybrid_clock.cc:648] HybridClock initialized: now 1786443081879963 us; error 39 us; skew 500 ppm
Aug 11 10:11:21 dist-test-slave-g350 krb5kdc[12009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443081, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:21.882869 12024 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:21.883455 12024 webserver.cc:460] Webserver started at http://127.22.177.62:42451/ using document root <none> and password file <none>
I20260811 10:11:21.883723 12024 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:21.883802 12024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:21.883970 12024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:21.885181 12024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8"
format_stamp: "Formatted at 2026-08-11 10:11:21 on dist-test-slave-g350"
server_key: "334c0283672fed9c85e602066b2cfb78"
server_key_iv: "c22cc15bf81b12ddaa057a058123b9a9"
server_key_version: "encryptionkey@0"
I20260811 10:11:21.885656 12024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8"
format_stamp: "Formatted at 2026-08-11 10:11:21 on dist-test-slave-g350"
server_key: "334c0283672fed9c85e602066b2cfb78"
server_key_iv: "c22cc15bf81b12ddaa057a058123b9a9"
server_key_version: "encryptionkey@0"
I20260811 10:11:21.887301 12024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:21.888257 12039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:21.888590 12024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:21.888733 12024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8"
format_stamp: "Formatted at 2026-08-11 10:11:21 on dist-test-slave-g350"
server_key: "334c0283672fed9c85e602066b2cfb78"
server_key_iv: "c22cc15bf81b12ddaa057a058123b9a9"
server_key_version: "encryptionkey@0"
I20260811 10:11:21.888821 12024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:21.912160 12024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:21.914577 12024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:21.914796 12024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:21.919714 12024 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:38145
I20260811 10:11:21.919749 12108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:38145 every 8 connection(s)
I20260811 10:11:21.920171 12024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:21.921080 12109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:21.923462 12109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: Bootstrap starting.
I20260811 10:11:21.924019 12109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:21.924261 12109 log.cc:824] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:21.925024 12109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: No bootstrap required, opened a new log
I20260811 10:11:21.926690 12109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38145 } }
I20260811 10:11:21.926841 12109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:21.926867 12109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d671b6a6f9d4341ba9993b2e14b7ed8, State: Initialized, Role: FOLLOWER
I20260811 10:11:21.927016 12109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38145 } }
I20260811 10:11:21.927101 12109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:21.927125 12109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:21.927155 12109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:21.927680 12109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38145 } }
I20260811 10:11:21.927769 12109 leader_election.cc:302] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d671b6a6f9d4341ba9993b2e14b7ed8; no voters:
I20260811 10:11:21.927902 12109 leader_election.cc:288] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:21.927990 12112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:21.928176 12112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d671b6a6f9d4341ba9993b2e14b7ed8, State: Running, Role: LEADER
I20260811 10:11:21.928234 12109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:21.928478 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12024
I20260811 10:11:21.928610 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:21.928850 23236 external_mini_cluster.cc:1428] Setting key 196628a94d05c7b6afcc282c4106d152
I20260811 10:11:21.929067 12112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38145 } }
I20260811 10:11:21.929772 12114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d671b6a6f9d4341ba9993b2e14b7ed8. Latest consensus state: current_term: 1 leader_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38145 } } }
I20260811 10:11:21.930086 12114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:21.929946 12113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d671b6a6f9d4341ba9993b2e14b7ed8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 38145 } } }
W20260811 10:11:21.930297 12129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 10:11:21.930322 12113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8 [sys.catalog]: This master's current role is: LEADER
W20260811 10:11:21.930351 12129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:21.930804 12132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:21.930981 12132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:21.932847 12132 catalog_manager.cc:1347] Generated new cluster ID: dda8c1015d704cab8bfd520b0ded3928
I20260811 10:11:21.932902 12132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:11:21 dist-test-slave-g350 krb5kdc[12009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:21.942226 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:21.929655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.22.177.62:38145)
0811 10:11:21.929915 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:21.929924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:21.929957 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:11:21.930083 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:21.930091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:21.930161 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:21.931128 (+ 967us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:21.931218 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:21.932440 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:21.933490 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:21.934116 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:21.934214 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:21.937168 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:21.937172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:21.937173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:21.937186 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:21.940035 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:21.940474 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:21.940478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:21.940478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:21.940601 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:21.940831 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:21.940832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:21.940832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:21.941172 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:21.941314 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:21.941827 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 10:11:21.941984 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":154,"threads_started":1}
I20260811 10:11:21.960233 12132 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:21.960783 12132 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:21.969707 12132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: Generated new TSK 0
I20260811 10:11:21.969933 12132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.177.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
Entry for principal kudu/127.22.177.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab.
I20260811 10:11:22.006058 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38145
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:22.094696 12137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:22.094949 12137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:22.094978 12137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:22.096385 12137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:22.096444 12137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:22.096475 12137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:11:22.098232 12137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.1.keytab
--principal=kudu/127.22.177.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:22.098557 12137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:22.098826 12137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:22.104611 12143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.104602 12144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.104604 12146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:22.104625 12137 server_base.cc:1034] running on GCE node
I20260811 10:11:22.105044 12137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:22.105335 12137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:22.106534 12137 hybrid_clock.cc:648] HybridClock initialized: now 1786443082106488 us; error 63 us; skew 500 ppm
Aug 11 10:11:22 dist-test-slave-g350 krb5kdc[12009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443082, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:22.109720 12137 init.cc:377] Logged in from keytab as kudu/127.22.177.1@KRBTEST.COM (short username kudu)
I20260811 10:11:22.110249 12137 webserver.cc:460] Webserver started at http://127.22.177.1:38943/ using document root <none> and password file <none>
I20260811 10:11:22.110528 12137 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:22.110607 12137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:22.110766 12137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:22.112010 12137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "dfc2bbb9a5aaf30eece040e38336b500"
server_key_iv: "928fc4f7ed97ade86d875971e27d93f4"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.112413 12137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "dfc2bbb9a5aaf30eece040e38336b500"
server_key_iv: "928fc4f7ed97ade86d875971e27d93f4"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.113935 12137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:22.114801 12154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.115101 12137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:22.115312 12137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "dfc2bbb9a5aaf30eece040e38336b500"
server_key_iv: "928fc4f7ed97ade86d875971e27d93f4"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.115404 12137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:22.125064 12137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:22.127326 12137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:22.127507 12137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:22.127756 12137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:22.128098 12137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:22.128153 12137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.128194 12137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:22.128223 12137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.134807 12137 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:43731
I20260811 10:11:22.134864 12288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:43731 every 8 connection(s)
I20260811 10:11:22.135195 12137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:11:22 dist-test-slave-g350 krb5kdc[12009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443082, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.1@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:22.142750 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12137
I20260811 10:11:22.142894 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:11:22.143163 23236 external_mini_cluster.cc:1428] Setting key f5e891938f80d924c6ca6ac9a91c9f2a
I20260811 10:11:22.145305 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.136072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:56001 (local address 127.22.177.62:38145)
0811 10:11:22.136137 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.136139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.136300 (+ 161us) server_negotiation.cc:407] Connection header received
0811 10:11:22.136829 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.136831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.136846 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.136908 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:22.137889 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.138399 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.139399 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.139595 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.139653 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.142061 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:22.142062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:22.142063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:22.142072 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:22.143808 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.144273 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.144274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.144275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.144342 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.144605 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.144607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.144607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.144731 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:22.144822 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.145111 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.145212 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:11:22.145571 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.135982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:38145 (local address 127.22.177.1:56001)
0811 10:11:22.136158 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.136203 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.136719 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.136959 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.136962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.137552 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:22.137775 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.137780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.138489 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.138490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.139314 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.139316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.139687 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.139687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.139798 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.140356 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:22.140364 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:22.141934 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:22.143929 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:22.143937 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:22.143939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:22.144168 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:22.144411 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:22.144412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:22.144413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:22.144532 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:22.144813 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:22.144821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:22.144966 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.145089 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.145266 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":108,"threads_started":1}
I20260811 10:11:22.146162 12289 heartbeater.cc:347] Connected to a master server at 127.22.177.62:38145
I20260811 10:11:22.146278 12289 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:22.146472 12289 heartbeater.cc:511] Master 127.22.177.62:38145 requested a full tablet report, sending...
I20260811 10:11:22.147126 12060 ts_manager.cc:194] Registered new tserver with Master: a0cfb0f2a4a145d7b53e9daa2deb9f02 (127.22.177.1:43731)
I20260811 10:11:22.147743 12060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.1@KRBTEST.COM'} at 127.22.177.1:56001
WARNING: no policy specified for kudu/127.22.177.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
Entry for principal kudu/127.22.177.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab.
I20260811 10:11:22.173317 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38145
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:22.262642 12296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:22.262923 12296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:22.262966 12296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:22.264256 12296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:22.264343 12296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:22.264392 12296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:11:22.265998 12296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.2.keytab
--principal=kudu/127.22.177.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:22.266309 12296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:22.266568 12296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:22.272225 12304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.272265 12306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.272225 12303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:22.272639 12296 server_base.cc:1034] running on GCE node
I20260811 10:11:22.272862 12296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:22.273134 12296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:22.274340 12296 hybrid_clock.cc:648] HybridClock initialized: now 1786443082274327 us; error 31 us; skew 500 ppm
Aug 11 10:11:22 dist-test-slave-g350 krb5kdc[12009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443082, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:22.277072 12296 init.cc:377] Logged in from keytab as kudu/127.22.177.2@KRBTEST.COM (short username kudu)
I20260811 10:11:22.277702 12296 webserver.cc:460] Webserver started at http://127.22.177.2:36387/ using document root <none> and password file <none>
I20260811 10:11:22.277980 12296 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:22.278051 12296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:22.278211 12296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:22.279292 12296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "baf9f5854cd44ea3884bf4313b55c523"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "1fa18a604a414f743a57254407b9cd12"
server_key_iv: "7685c089bb39e5168d43e048d3bfcc21"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.279677 12296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "baf9f5854cd44ea3884bf4313b55c523"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "1fa18a604a414f743a57254407b9cd12"
server_key_iv: "7685c089bb39e5168d43e048d3bfcc21"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.281433 12296 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:11:22.282361 12314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.282575 12296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:22.282691 12296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "baf9f5854cd44ea3884bf4313b55c523"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "1fa18a604a414f743a57254407b9cd12"
server_key_iv: "7685c089bb39e5168d43e048d3bfcc21"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.282771 12296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:22.299304 12296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:22.301846 12296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:22.302002 12296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:22.302274 12296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:22.302616 12296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:22.302651 12296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.302675 12296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:22.302722 12296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.308655 12296 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:44589
I20260811 10:11:22.308756 12441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:44589 every 8 connection(s)
I20260811 10:11:22.309095 12296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:11:22.309397 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12296
I20260811 10:11:22.309528 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:11:22.309768 23236 external_mini_cluster.cc:1428] Setting key 358ba04a606b655e107d0f6e2d93e738
Aug 11 10:11:22 dist-test-slave-g350 krb5kdc[12009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443082, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.2@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.3@KRBTEST.COM; defaulting to no policy
I20260811 10:11:22.322556 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.309968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:35741 (local address 127.22.177.62:38145)
0811 10:11:22.310020 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.310024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.310294 (+ 270us) server_negotiation.cc:407] Connection header received
0811 10:11:22.311837 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.311838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.311848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.311892 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:22.313079 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.313944 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.315235 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.315658 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.315769 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.318907 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:22.318909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:22.318910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:22.318920 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:22.320807 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.321324 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.321326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.321327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.321394 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.321697 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.321697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.321698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.321816 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:22.321870 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.322375 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.322468 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:11:22.322724 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.309956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:38145 (local address 127.22.177.2:35741)
0811 10:11:22.310144 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.310196 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.311706 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.312105 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.312110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.312594 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:22.312937 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.312944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.314076 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.314076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.315118 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.315121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.315811 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.315813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.315977 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.316650 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:22.316659 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:22.318786 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:22.320912 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:22.320919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:22.320921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:22.321238 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:22.321470 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:22.321471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:22.321471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:22.321582 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:22.321897 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:22.321905 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:22.322122 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.322305 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.322466 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260811 10:11:22.323279 12442 heartbeater.cc:347] Connected to a master server at 127.22.177.62:38145
I20260811 10:11:22.323385 12442 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:22.323683 12442 heartbeater.cc:511] Master 127.22.177.62:38145 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.3@KRBTEST.COM" created.
I20260811 10:11:22.324172 12060 ts_manager.cc:194] Registered new tserver with Master: baf9f5854cd44ea3884bf4313b55c523 (127.22.177.2:44589)
I20260811 10:11:22.324572 12060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.2@KRBTEST.COM'} at 127.22.177.2:35741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
Entry for principal kudu/127.22.177.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab.
I20260811 10:11:22.341663 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38145
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:43059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:22.427251 12450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:22.427476 12450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:22.427502 12450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:22.428681 12450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:22.428725 12450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:22.428779 12450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:11:22.430526 12450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:43059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.3.keytab
--principal=kudu/127.22.177.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:38145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:22.430730 12450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:22.430922 12450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:22.436156 12460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.436146 12455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.436193 12457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:22.436508 12450 server_base.cc:1034] running on GCE node
I20260811 10:11:22.436693 12450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:22.436898 12450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:22.438067 12450 hybrid_clock.cc:648] HybridClock initialized: now 1786443082438054 us; error 38 us; skew 500 ppm
Aug 11 10:11:22 dist-test-slave-g350 krb5kdc[12009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443082, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:22.440729 12450 init.cc:377] Logged in from keytab as kudu/127.22.177.3@KRBTEST.COM (short username kudu)
I20260811 10:11:22.441187 12450 webserver.cc:460] Webserver started at http://127.22.177.3:35149/ using document root <none> and password file <none>
I20260811 10:11:22.441469 12450 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:22.441545 12450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:22.441726 12450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:22.442831 12450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "752c261e6d1d4dd6acb9f445338b5fac"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "d04f34b596acbabb4f2e0bb9b6e248aa"
server_key_iv: "a04fcb6f1ac4781ef49103e468f3daca"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.443200 12450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "752c261e6d1d4dd6acb9f445338b5fac"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "d04f34b596acbabb4f2e0bb9b6e248aa"
server_key_iv: "a04fcb6f1ac4781ef49103e468f3daca"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.444669 12450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:22.445530 12467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.445750 12450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:22.445866 12450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "752c261e6d1d4dd6acb9f445338b5fac"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "d04f34b596acbabb4f2e0bb9b6e248aa"
server_key_iv: "a04fcb6f1ac4781ef49103e468f3daca"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.445945 12450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:22.462270 12450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:22.464576 12450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:22.464761 12450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:22.465036 12450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:22.465377 12450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:22.465433 12450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.465471 12450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:22.465517 12450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.471926 12450 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:36967
I20260811 10:11:22.472003 12597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:36967 every 8 connection(s)
I20260811 10:11:22.472345 12450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:11:22.478048 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12450
I20260811 10:11:22.478170 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:11:22.478364 23236 external_mini_cluster.cc:1428] Setting key fa651e9fbc869091650421939cc86280
Aug 11 10:11:22 dist-test-slave-g350 krb5kdc[12009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443082, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.3@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:22.482502 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.473169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:33725 (local address 127.22.177.62:38145)
0811 10:11:22.473320 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.473323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.473467 (+ 144us) server_negotiation.cc:407] Connection header received
0811 10:11:22.474120 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.474121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.474135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.474193 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:22.475186 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.475708 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.476736 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.476944 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.477034 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.479683 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:22.479684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:22.479684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:22.479691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:22.481138 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.481580 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.481581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.481581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.481715 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.481962 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.481963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.481963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.482079 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:22.482132 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.482358 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.482435 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:11:22.482770 12602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.473128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:38145 (local address 127.22.177.3:33725)
0811 10:11:22.473317 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.473374 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.473996 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.474261 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.474265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.474880 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:22.475079 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.475085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.475813 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.475814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.476639 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.476641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.477048 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.477049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.477174 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.477776 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:22.477784 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:22.479592 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:22.481255 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:22.481263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:22.481265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:22.481507 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:22.481781 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:22.481781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:22.481782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:22.481899 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:22.482121 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:22.482124 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:22.482243 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.482329 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.482476 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260811 10:11:22.483371 12599 heartbeater.cc:347] Connected to a master server at 127.22.177.62:38145
I20260811 10:11:22.483464 12599 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:22.483614 12599 heartbeater.cc:511] Master 127.22.177.62:38145 requested a full tablet report, sending...
I20260811 10:11:22.484112 12060 ts_manager.cc:194] Registered new tserver with Master: 752c261e6d1d4dd6acb9f445338b5fac (127.22.177.3:36967)
I20260811 10:11:22.484525 12060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.177.3@KRBTEST.COM'} at 127.22.177.3:33725
I20260811 10:11:22.490556 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:11:22.498292 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.491979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.22.177.62:38145)
0811 10:11:22.492023 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.492025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.492126 (+ 101us) server_negotiation.cc:407] Connection header received
0811 10:11:22.492185 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.492186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.492195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.492254 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:22.492795 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.493492 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.494250 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.494462 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.494521 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.495780 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:22.495781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:22.495782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:22.495788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:22.497122 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.497473 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.497474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.497474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.497534 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:22.497788 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:22.497789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:22.497789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:22.497951 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:22.498041 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.498140 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.498210 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:22.500099 12060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:11:22.501976 12060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:11:22.508864 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.504534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.22.177.2:44589)
0811 10:11:22.504747 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.504752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.505031 (+ 279us) server_negotiation.cc:407] Connection header received
0811 10:11:22.505087 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.505089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.505108 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.505197 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.505401 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.506265 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.507555 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.508408 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.508494 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.508559 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.508728 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.508772 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260811 10:11:22.508852 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.504606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43731 (local address 127.0.0.1:54510)
0811 10:11:22.504942 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.504953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.504996 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.505301 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.505302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.505304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.505378 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.505380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.506914 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.506915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.507861 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.507863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.508647 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.508648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.508710 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.508713 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.508757 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.508785 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":83,"threads_started":1}
I20260811 10:11:22.508853 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.504359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44589 (local address 127.0.0.1:52180)
0811 10:11:22.504752 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.504799 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.504924 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.505208 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.505211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.505213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.505293 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.505295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.506370 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.506371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.507448 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.507451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.508509 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.508510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.508605 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.508620 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.508695 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.508765 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":88,"threads_started":1}
I20260811 10:11:22.510030 12616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.504806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:36967 (local address 127.0.0.1:51862)
0811 10:11:22.505046 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.505057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.505382 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.505662 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.505663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.505665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.505712 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.505713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.507167 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.507168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.508524 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.508528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.509764 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.509765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.509833 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.509838 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.509900 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.509950 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":79,"threads_started":1}
I20260811 10:11:22.510043 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.504927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.22.177.3:36967)
0811 10:11:22.505119 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.505123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.505455 (+ 332us) server_negotiation.cc:407] Connection header received
0811 10:11:22.505509 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.505511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.505532 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.505643 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.505778 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.506951 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.508627 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.509676 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.509761 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.509823 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.509912 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.509962 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260811 10:11:22.509999 12364 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2d6a30639c4690a956a74849c45e00 (DEFAULT_TABLE table=test-table [id=f590b9ff2c0d4b84ac3cab93b9398bb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:22.510404 12364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2d6a30639c4690a956a74849c45e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:22.510998 12519 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2d6a30639c4690a956a74849c45e00 (DEFAULT_TABLE table=test-table [id=f590b9ff2c0d4b84ac3cab93b9398bb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:22.511344 12519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2d6a30639c4690a956a74849c45e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:22.509438 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.504757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54510 (local address 127.22.177.1:43731)
0811 10:11:22.504973 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.504978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.504991 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:11:22.505115 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.505118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.505137 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.505210 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.505483 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.506795 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.507953 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.508475 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.508549 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.508613 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.509305 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.509351 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
I20260811 10:11:22.513773 12621 tablet_bootstrap.cc:492] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Bootstrap starting.
I20260811 10:11:22.514212 12213 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2d6a30639c4690a956a74849c45e00 (DEFAULT_TABLE table=test-table [id=f590b9ff2c0d4b84ac3cab93b9398bb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:22.514564 12213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2d6a30639c4690a956a74849c45e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:22.514706 12622 tablet_bootstrap.cc:492] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Bootstrap starting.
I20260811 10:11:22.514815 12621 tablet_bootstrap.cc:654] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:22.515370 12622 tablet_bootstrap.cc:654] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:22.515462 12621 log.cc:824] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:22.516019 12622 log.cc:824] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:22.516417 12621 tablet_bootstrap.cc:492] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: No bootstrap required, opened a new log
I20260811 10:11:22.516510 12621 ts_tablet_manager.cc:1397] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:11:22.516937 12622 tablet_bootstrap.cc:492] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: No bootstrap required, opened a new log
I20260811 10:11:22.517069 12622 ts_tablet_manager.cc:1397] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:11:22.518440 12626 tablet_bootstrap.cc:492] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: Bootstrap starting.
I20260811 10:11:22.518491 12621 raft_consensus.cc:348] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.518697 12621 raft_consensus.cc:374] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:22.518756 12621 raft_consensus.cc:729] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf9f5854cd44ea3884bf4313b55c523, State: Initialized, Role: FOLLOWER
I20260811 10:11:22.518926 12621 consensus_queue.cc:260] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.519256 12442 heartbeater.cc:503] Master 127.22.177.62:38145 was elected leader, sending a full tablet report...
I20260811 10:11:22.519354 12626 tablet_bootstrap.cc:654] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:22.519333 12621 ts_tablet_manager.cc:1428] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:22.519479 12622 raft_consensus.cc:348] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.519717 12622 raft_consensus.cc:374] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:22.519761 12622 raft_consensus.cc:729] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752c261e6d1d4dd6acb9f445338b5fac, State: Initialized, Role: FOLLOWER
I20260811 10:11:22.519870 12622 consensus_queue.cc:260] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.520002 12626 log.cc:824] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:22.520143 12622 ts_tablet_manager.cc:1428] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:11:22.520181 12599 heartbeater.cc:503] Master 127.22.177.62:38145 was elected leader, sending a full tablet report...
I20260811 10:11:22.520833 12626 tablet_bootstrap.cc:492] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: No bootstrap required, opened a new log
I20260811 10:11:22.520915 12626 ts_tablet_manager.cc:1397] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:11:22.522603 12626 raft_consensus.cc:348] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.522763 12626 raft_consensus.cc:374] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:22.522812 12626 raft_consensus.cc:729] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cfb0f2a4a145d7b53e9daa2deb9f02, State: Initialized, Role: FOLLOWER
I20260811 10:11:22.522933 12626 consensus_queue.cc:260] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.523198 12626 ts_tablet_manager.cc:1428] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:11:22.523262 12289 heartbeater.cc:503] Master 127.22.177.62:38145 was elected leader, sending a full tablet report...
I20260811 10:11:22.553385 12632 raft_consensus.cc:482] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:22.553568 12632 raft_consensus.cc:504] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.554044 12632 leader_election.cc:288] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baf9f5854cd44ea3884bf4313b55c523 (127.22.177.2:44589), 752c261e6d1d4dd6acb9f445338b5fac (127.22.177.3:36967)
I20260811 10:11:22.556468 12629 raft_consensus.cc:482] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:22.556620 12629 raft_consensus.cc:504] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.557057 12629 leader_election.cc:288] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baf9f5854cd44ea3884bf4313b55c523 (127.22.177.2:44589), a0cfb0f2a4a145d7b53e9daa2deb9f02 (127.22.177.1:43731)
I20260811 10:11:22.557495 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.554283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44589 (local address 127.22.177.1:37613)
0811 10:11:22.554406 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.554420 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.554506 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.554751 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.554752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.554754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.554845 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.554846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.555583 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.555584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.556486 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.556489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.557227 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.557228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.557323 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.557327 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.557376 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.557418 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 10:11:22.558022 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.554546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:37613 (local address 127.22.177.2:44589)
0811 10:11:22.554614 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.554616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.554625 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:11:22.554655 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.554656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.554666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.554722 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.554930 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.555481 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.556581 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.557101 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.557178 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.557188 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.557893 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.557949 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:11:22.558107 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.554566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:36565 (local address 127.22.177.3:36967)
0811 10:11:22.554636 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.554639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.554753 (+ 114us) server_negotiation.cc:407] Connection header received
0811 10:11:22.554825 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.554827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.554838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.554923 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.555168 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.555723 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.557057 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.557678 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.557745 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.557755 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.557989 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.558032 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:11:22.558310 12633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.554467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:36967 (local address 127.22.177.1:36565)
0811 10:11:22.554666 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.554678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.554760 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.554927 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.554928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.554930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.555028 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.555030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.555822 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.555823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.556945 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.556949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.557803 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.557803 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.557893 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.557898 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.558190 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.558234 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":144,"threads_started":1}
I20260811 10:11:22.558352 12388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d6a30639c4690a956a74849c45e00" candidate_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf9f5854cd44ea3884bf4313b55c523" is_pre_election: true
I20260811 10:11:22.558553 12388 raft_consensus.cc:2454] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cfb0f2a4a145d7b53e9daa2deb9f02 in term 0.
I20260811 10:11:22.558830 12157 leader_election.cc:302] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0cfb0f2a4a145d7b53e9daa2deb9f02, baf9f5854cd44ea3884bf4313b55c523; no voters:
I20260811 10:11:22.558979 12632 raft_consensus.cc:2781] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:11:22.559063 12632 raft_consensus.cc:482] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:11:22.559111 12632 raft_consensus.cc:3037] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:22.559144 12545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d6a30639c4690a956a74849c45e00" candidate_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "752c261e6d1d4dd6acb9f445338b5fac" is_pre_election: true
I20260811 10:11:22.559250 12545 raft_consensus.cc:2454] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cfb0f2a4a145d7b53e9daa2deb9f02 in term 0.
W20260811 10:11:22.559823 12445 tablet.cc:2369] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:11:22.560220 12632 raft_consensus.cc:504] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.560372 12632 leader_election.cc:288] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [CANDIDATE]: Term 1 election: Requested vote from peers baf9f5854cd44ea3884bf4313b55c523 (127.22.177.2:44589), 752c261e6d1d4dd6acb9f445338b5fac (127.22.177.3:36967)
I20260811 10:11:22.560585 12545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d6a30639c4690a956a74849c45e00" candidate_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "752c261e6d1d4dd6acb9f445338b5fac"
I20260811 10:11:22.560559 12388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d6a30639c4690a956a74849c45e00" candidate_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf9f5854cd44ea3884bf4313b55c523"
I20260811 10:11:22.560660 12388 raft_consensus.cc:3037] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:22.560660 12545 raft_consensus.cc:3037] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:22.561523 12602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.557192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:44589 (local address 127.22.177.3:45491)
0811 10:11:22.557463 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.557476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.557533 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.557819 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.557820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.557821 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.557886 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.557889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.559077 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.559078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.560391 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.560394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.561247 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.561248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.561330 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.561335 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.561381 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.561430 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260811 10:11:22.561700 12545 raft_consensus.cc:2454] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cfb0f2a4a145d7b53e9daa2deb9f02 in term 1.
I20260811 10:11:22.561831 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.557259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:45491 (local address 127.22.177.2:44589)
0811 10:11:22.557663 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.557665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.557673 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:22.557699 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.557700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.557712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.557976 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.558013 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.558969 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.560507 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.561150 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.561664 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.561673 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.561694 (+ 21us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.561746 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":151,"threads_started":1}
I20260811 10:11:22.561862 12388 raft_consensus.cc:2454] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cfb0f2a4a145d7b53e9daa2deb9f02 in term 1.
I20260811 10:11:22.561894 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.557715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:53009 (local address 127.22.177.1:43731)
0811 10:11:22.557752 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.557754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.557762 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:22.557791 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.557792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.557801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.557849 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:22.558458 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.559237 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.560869 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.561447 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.561490 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.561498 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.561770 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.561807 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:22.561970 12157 leader_election.cc:302] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 752c261e6d1d4dd6acb9f445338b5fac, a0cfb0f2a4a145d7b53e9daa2deb9f02; no voters:
I20260811 10:11:22.562100 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.557324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43731 (local address 127.22.177.3:53009)
0811 10:11:22.557476 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.557486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.557533 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.557906 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.557907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.557914 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:22.557990 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.557991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.559511 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.559512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.560759 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.560761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.561563 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.561565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.561674 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.561679 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.561971 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.562022 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260811 10:11:22.562075 12388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d6a30639c4690a956a74849c45e00" candidate_uuid: "752c261e6d1d4dd6acb9f445338b5fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf9f5854cd44ea3884bf4313b55c523" is_pre_election: true
I20260811 10:11:22.562176 12388 raft_consensus.cc:2379] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 752c261e6d1d4dd6acb9f445338b5fac in current term 1: Already voted for candidate a0cfb0f2a4a145d7b53e9daa2deb9f02 in this term.
I20260811 10:11:22.562192 12632 raft_consensus.cc:2781] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:22.562374 12632 raft_consensus.cc:686] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 1 LEADER]: Becoming Leader. State: Replica: a0cfb0f2a4a145d7b53e9daa2deb9f02, State: Running, Role: LEADER
I20260811 10:11:22.562491 12632 consensus_queue.cc:237] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } }
I20260811 10:11:22.562582 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d6a30639c4690a956a74849c45e00" candidate_uuid: "752c261e6d1d4dd6acb9f445338b5fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" is_pre_election: true
I20260811 10:11:22.562996 12470 leader_election.cc:302] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 752c261e6d1d4dd6acb9f445338b5fac; no voters: a0cfb0f2a4a145d7b53e9daa2deb9f02, baf9f5854cd44ea3884bf4313b55c523
I20260811 10:11:22.563174 12629 raft_consensus.cc:2726] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 10:11:22.563568 12058 catalog_manager.cc:5495] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0cfb0f2a4a145d7b53e9daa2deb9f02 (127.22.177.1). New cstate: current_term: 1 leader_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0cfb0f2a4a145d7b53e9daa2deb9f02" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 43731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 } health_report { overall_health: UNKNOWN } } }
I20260811 10:11:22.570537 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.567703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.22.177.1:43731)
0811 10:11:22.567787 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.567789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.567798 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:11:22.567884 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.567885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.567904 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.567981 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:22.568152 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.568671 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.569534 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.569875 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.569947 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.570132 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:22.570310 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:22.570387 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.570424 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.570444 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 10:11:22.572606 12388 raft_consensus.cc:1261] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 1 FOLLOWER]: Refusing update from remote peer a0cfb0f2a4a145d7b53e9daa2deb9f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:22.572654 12545 raft_consensus.cc:1261] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 1 FOLLOWER]: Refusing update from remote peer a0cfb0f2a4a145d7b53e9daa2deb9f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:22.573017 12632 consensus_queue.cc:1035] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "752c261e6d1d4dd6acb9f445338b5fac" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 36967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:22.573189 12636 consensus_queue.cc:1035] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf9f5854cd44ea3884bf4313b55c523" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 44589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:22.575786 12639 mvcc.cc:205] Tried to move back new op lower bound from 7317270866214027264 to 7317270866177048576. Current Snapshot: MvccSnapshot[applied={T|T < 7317270866214027264}]
I20260811 10:11:22.576282 12641 mvcc.cc:205] Tried to move back new op lower bound from 7317270866214027264 to 7317270866177048576. Current Snapshot: MvccSnapshot[applied={T|T < 7317270866214027264}]
I20260811 10:11:22.579382 12060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:22.579473 12060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:22.580374 12060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d671b6a6f9d4341ba9993b2e14b7ed8: Sending DeleteTablet for 3 replicas of tablet 7a2d6a30639c4690a956a74849c45e00
I20260811 10:11:22.580824 12213 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a2d6a30639c4690a956a74849c45e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:22 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:54510
I20260811 10:11:22.580864 12519 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a2d6a30639c4690a956a74849c45e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:22 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:51862
I20260811 10:11:22.581065 12364 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a2d6a30639c4690a956a74849c45e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:22 UTC) from {username='kudu', principal='kudu/127.22.177.62@KRBTEST.COM'} at 127.0.0.1:52180
I20260811 10:11:22.581101 12653 tablet_replica.cc:330] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02: stopping tablet replica
I20260811 10:11:22.581221 12653 raft_consensus.cc:2229] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:11:22.581331 12653 raft_consensus.cc:2258] T 7a2d6a30639c4690a956a74849c45e00 P a0cfb0f2a4a145d7b53e9daa2deb9f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:22.581461 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12137
I20260811 10:11:22.581490 12655 tablet_replica.cc:330] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: stopping tablet replica
I20260811 10:11:22.582937 12655 raft_consensus.cc:2229] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:22.583112 12655 raft_consensus.cc:2258] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:22.583449 12654 tablet_replica.cc:330] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: stopping tablet replica
I20260811 10:11:22.583570 12654 raft_consensus.cc:2229] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:22.583683 12654 raft_consensus.cc:2258] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:22.584041 12655 ts_tablet_manager.cc:1905] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:22.584295 12654 ts_tablet_manager.cc:1905] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:22.586202 12654 ts_tablet_manager.cc:1918] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:22.586210 12655 ts_tablet_manager.cc:1918] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:22.586365 12655 log.cc:1196] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/7a2d6a30639c4690a956a74849c45e00
I20260811 10:11:22.586378 12654 log.cc:1196] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/7a2d6a30639c4690a956a74849c45e00
I20260811 10:11:22.586689 12654 ts_tablet_manager.cc:1939] T 7a2d6a30639c4690a956a74849c45e00 P 752c261e6d1d4dd6acb9f445338b5fac: Deleting consensus metadata
I20260811 10:11:22.586882 12655 ts_tablet_manager.cc:1939] T 7a2d6a30639c4690a956a74849c45e00 P baf9f5854cd44ea3884bf4313b55c523: Deleting consensus metadata
I20260811 10:11:22.587226 12043 catalog_manager.cc:4841] TS 752c261e6d1d4dd6acb9f445338b5fac (127.22.177.3:36967): tablet 7a2d6a30639c4690a956a74849c45e00 (table test-table [id=f590b9ff2c0d4b84ac3cab93b9398bb8]) successfully deleted
I20260811 10:11:22.587357 12043 catalog_manager.cc:4841] TS baf9f5854cd44ea3884bf4313b55c523 (127.22.177.2:44589): tablet 7a2d6a30639c4690a956a74849c45e00 (table test-table [id=f590b9ff2c0d4b84ac3cab93b9398bb8]) successfully deleted
W20260811 10:11:22.590044 12043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:43731 (error 108)
I20260811 10:11:22.590363 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12296
I20260811 10:11:22.598795 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.598594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:43731 (local address 127.0.0.1:54524)
0811 10:11:22.598676 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.598741 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:43731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 10:11:22.598974 12043 catalog_manager.cc:4584] TS a0cfb0f2a4a145d7b53e9daa2deb9f02 (127.22.177.1:43731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a2d6a30639c4690a956a74849c45e00: Network error: Client connection negotiation failed: client connection to 127.22.177.1:43731: connect: Connection refused (error 111)
I20260811 10:11:22.602838 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12450
I20260811 10:11:22.610299 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12024
2026-08-11T10:11:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2179 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T10:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:22Z Disabled control of system clock
I20260811 10:11:22.644053 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:44633
--webserver_interface=127.22.177.62
--webserver_port=0
--builtin_ntp_servers=127.22.177.20:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:44633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:22.731006 12660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:22.731254 12660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:22.731279 12660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:22.732565 12660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:22.732620 12660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:22.732633 12660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:22.732646 12660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:22.732658 12660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:22.734192 12660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:44633
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:44633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:22.734494 12660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:22.734831 12660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:22.740289 12669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.740289 12671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.740643 12668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:22.740643 12660 server_base.cc:1034] running on GCE node
I20260811 10:11:22.740964 12660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:22.741267 12660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:22.742498 12660 hybrid_clock.cc:648] HybridClock initialized: now 1786443082742467 us; error 38 us; skew 500 ppm
I20260811 10:11:22.743908 12660 webserver.cc:460] Webserver started at http://127.22.177.62:33237/ using document root <none> and password file <none>
I20260811 10:11:22.744194 12660 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:22.744268 12660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:22.744452 12660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:22.745687 12660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "d82746217a7d46ccba7bc355325d2e30"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "e6d75be7c6dd7effc8843274dff00b46"
server_key_iv: "62448142c0ea5c2a0f6b9c166ed77ef8"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.746094 12660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "d82746217a7d46ccba7bc355325d2e30"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "e6d75be7c6dd7effc8843274dff00b46"
server_key_iv: "62448142c0ea5c2a0f6b9c166ed77ef8"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.747475 12660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:11:22.748291 12677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.748576 12660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:22.748718 12660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "d82746217a7d46ccba7bc355325d2e30"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "e6d75be7c6dd7effc8843274dff00b46"
server_key_iv: "62448142c0ea5c2a0f6b9c166ed77ef8"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.748813 12660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:22.763574 12660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:22.763972 12660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:22.764149 12660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:22.768584 12660 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:44633
I20260811 10:11:22.768640 12736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:44633 every 8 connection(s)
I20260811 10:11:22.769037 12660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:22.769532 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12660
I20260811 10:11:22.769680 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:22.769922 23236 external_mini_cluster.cc:1428] Setting key ccfd71cdecf754d5e2ae185ef5da216c
I20260811 10:11:22.770349 12737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:22.773945 12737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30: Bootstrap starting.
I20260811 10:11:22.774644 12737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:22.774960 12737 log.cc:824] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:22.775930 12737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30: No bootstrap required, opened a new log
I20260811 10:11:22.776990 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.770511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48086 (local address 127.22.177.62:44633)
0811 10:11:22.770766 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.770775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.770818 (+ 43us) server_negotiation.cc:407] Connection header received
0811 10:11:22.771526 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.771536 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.771623 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.771787 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:22.772088 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.773438 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.774465 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.775112 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.775247 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.776216 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:22.776220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:22.776221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:22.776426 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:22.776475 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:22.776491 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:22.776565 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.776651 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.776737 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":111,"threads_started":1}
I20260811 10:11:22.778373 12737 raft_consensus.cc:348] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82746217a7d46ccba7bc355325d2e30" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 44633 } }
I20260811 10:11:22.778509 12737 raft_consensus.cc:374] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:22.778532 12737 raft_consensus.cc:729] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82746217a7d46ccba7bc355325d2e30, State: Initialized, Role: FOLLOWER
I20260811 10:11:22.778680 12737 consensus_queue.cc:260] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82746217a7d46ccba7bc355325d2e30" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 44633 } }
I20260811 10:11:22.778757 12737 raft_consensus.cc:388] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:22.778781 12737 raft_consensus.cc:482] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:22.778811 12737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:22.779423 12737 raft_consensus.cc:504] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82746217a7d46ccba7bc355325d2e30" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 44633 } }
I20260811 10:11:22.779515 12737 leader_election.cc:302] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d82746217a7d46ccba7bc355325d2e30; no voters:
I20260811 10:11:22.779721 12737 leader_election.cc:288] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:22.779788 12743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:22.779976 12743 raft_consensus.cc:686] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [term 1 LEADER]: Becoming Leader. State: Replica: d82746217a7d46ccba7bc355325d2e30, State: Running, Role: LEADER
I20260811 10:11:22.780014 12737 sys_catalog.cc:564] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:22.780292 12743 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82746217a7d46ccba7bc355325d2e30" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 44633 } }
I20260811 10:11:22.780784 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d82746217a7d46ccba7bc355325d2e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82746217a7d46ccba7bc355325d2e30" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 44633 } } }
I20260811 10:11:22.780790 12746 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82746217a7d46ccba7bc355325d2e30. Latest consensus state: current_term: 1 leader_uuid: "d82746217a7d46ccba7bc355325d2e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82746217a7d46ccba7bc355325d2e30" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 44633 } } }
I20260811 10:11:22.780897 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:22.780910 12746 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:22.781109 12754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:22.781689 12754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:22.783185 12754 catalog_manager.cc:1347] Generated new cluster ID: b9c79689c2604f6abab756e806f54813
I20260811 10:11:22.783253 12754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:22.791437 12754 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:22.791985 12754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:22.794854 12754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30: Generated new TSK 0
I20260811 10:11:22.795034 12754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:22.802873 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.1:0
--local_ip_for_outbound_sockets=127.22.177.1
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:44633
--builtin_ntp_servers=127.22.177.20:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:11:22.893146 12769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:22.893404 12769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:22.893427 12769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:22.894670 12769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:22.894716 12769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:22.894742 12769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.1
I20260811 10:11:22.896174 12769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.177.1
--webserver_port=0
--tserver_master_addrs=127.22.177.62:44633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.1
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:22.896430 12769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:22.896764 12769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:22.902770 12774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.902704 12775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:22.903033 12777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:22.903013 12769 server_base.cc:1034] running on GCE node
I20260811 10:11:22.903399 12769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:22.903676 12769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:22.904891 12769 hybrid_clock.cc:648] HybridClock initialized: now 1786443082904864 us; error 41 us; skew 500 ppm
I20260811 10:11:22.906648 12769 webserver.cc:460] Webserver started at http://127.22.177.1:35963/ using document root <none> and password file <none>
I20260811 10:11:22.906965 12769 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:22.907058 12769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:22.907240 12769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:22.908490 12769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data/instance:
uuid: "94c3f7074cce43b89f5c5f3c77953549"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "84e211ad0e826b8eeae4e8a68864d177"
server_key_iv: "c835e399ba2a1a41d2ca2ebfb13fe169"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.908931 12769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance:
uuid: "94c3f7074cce43b89f5c5f3c77953549"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "84e211ad0e826b8eeae4e8a68864d177"
server_key_iv: "c835e399ba2a1a41d2ca2ebfb13fe169"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.910687 12769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:22.911630 12783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.911821 12769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:22.911934 12769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal
uuid: "94c3f7074cce43b89f5c5f3c77953549"
format_stamp: "Formatted at 2026-08-11 10:11:22 on dist-test-slave-g350"
server_key: "84e211ad0e826b8eeae4e8a68864d177"
server_key_iv: "c835e399ba2a1a41d2ca2ebfb13fe169"
server_key_version: "encryptionkey@0"
I20260811 10:11:22.912026 12769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:22.927026 12769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:22.927395 12769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:22.927558 12769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:22.927845 12769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:22.928220 12769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:22.928275 12769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.928323 12769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:22.928354 12769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:22.934403 12769 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.1:33145
I20260811 10:11:22.934511 12910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.1:33145 every 8 connection(s)
I20260811 10:11:22.934841 12769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/data/info.pb
I20260811 10:11:22.941105 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.935799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.1:55389 (local address 127.22.177.62:44633)
0811 10:11:22.935881 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:22.935883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:22.936122 (+ 239us) server_negotiation.cc:407] Connection header received
0811 10:11:22.936666 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:22.936667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:22.936681 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:22.936797 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:22.937540 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.938147 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.939299 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.939550 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.939619 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.940774 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:22.940776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:22.940777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:22.940783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:22.940801 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:22.940805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:22.940846 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:22.940949 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 10:11:22.941006 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:11:22.941298 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:22.935758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:44633 (local address 127.22.177.1:55389)
0811 10:11:22.935982 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:22.936026 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:22.936567 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:22.936887 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:22.936890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:22.936899 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:22.937441 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.937447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.938252 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.938253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.939212 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:22.939216 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:22.939739 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:22.939740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:22.939921 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:22.940603 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:22.940604 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:22.940633 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:22.940677 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:22.940860 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:22.940882 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:22.940886 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:22.940951 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:11:22.941062 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":132,"threads_started":1}
I20260811 10:11:22.941725 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12769
I20260811 10:11:22.941854 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-0/wal/instance
I20260811 10:11:22.942090 23236 external_mini_cluster.cc:1428] Setting key aec83b8724a841a4c0cec28ca24efb5d
I20260811 10:11:22.943399 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.2:0
--local_ip_for_outbound_sockets=127.22.177.2
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:44633
--builtin_ntp_servers=127.22.177.20:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:11:22.944747 12911 heartbeater.cc:347] Connected to a master server at 127.22.177.62:44633
I20260811 10:11:22.944890 12911 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:22.945145 12911 heartbeater.cc:511] Master 127.22.177.62:44633 requested a full tablet report, sending...
I20260811 10:11:22.945976 12697 ts_manager.cc:194] Registered new tserver with Master: 94c3f7074cce43b89f5c5f3c77953549 (127.22.177.1:33145)
I20260811 10:11:22.946974 12697 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.1:55389
W20260811 10:11:23.048218 12914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:23.048477 12914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:23.048516 12914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:23.049726 12914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:23.049800 12914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:23.049842 12914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.2
I20260811 10:11:23.051250 12914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.177.2
--webserver_port=0
--tserver_master_addrs=127.22.177.62:44633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.2
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:23.051541 12914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:23.051867 12914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:23.057611 12924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:23.057585 12921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:23.057792 12914 server_base.cc:1034] running on GCE node
W20260811 10:11:23.057971 12922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:23.058317 12914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:23.058547 12914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:23.059734 12914 hybrid_clock.cc:648] HybridClock initialized: now 1786443083059719 us; error 31 us; skew 500 ppm
I20260811 10:11:23.061203 12914 webserver.cc:460] Webserver started at http://127.22.177.2:45827/ using document root <none> and password file <none>
I20260811 10:11:23.061548 12914 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:23.061658 12914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:23.061834 12914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:23.063082 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data/instance:
uuid: "f1d9344d63074fa7bd3461465379a9d4"
format_stamp: "Formatted at 2026-08-11 10:11:23 on dist-test-slave-g350"
server_key: "1b83b280c432fe3334a42bc59c0e5bf0"
server_key_iv: "9b1eb8ce62d78c2d761d109c250e6a38"
server_key_version: "encryptionkey@0"
I20260811 10:11:23.063493 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance:
uuid: "f1d9344d63074fa7bd3461465379a9d4"
format_stamp: "Formatted at 2026-08-11 10:11:23 on dist-test-slave-g350"
server_key: "1b83b280c432fe3334a42bc59c0e5bf0"
server_key_iv: "9b1eb8ce62d78c2d761d109c250e6a38"
server_key_version: "encryptionkey@0"
I20260811 10:11:23.064960 12914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:23.065937 12931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:23.066155 12914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:11:23.066327 12914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal
uuid: "f1d9344d63074fa7bd3461465379a9d4"
format_stamp: "Formatted at 2026-08-11 10:11:23 on dist-test-slave-g350"
server_key: "1b83b280c432fe3334a42bc59c0e5bf0"
server_key_iv: "9b1eb8ce62d78c2d761d109c250e6a38"
server_key_version: "encryptionkey@0"
I20260811 10:11:23.066422 12914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:23.090673 12914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:23.091118 12914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:23.091362 12914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:23.091643 12914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:23.092031 12914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:23.092093 12914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:23.092142 12914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:23.092175 12914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:23.099067 12914 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.2:41819
I20260811 10:11:23.099154 13060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.2:41819 every 8 connection(s)
I20260811 10:11:23.099536 12914 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/data/info.pb
I20260811 10:11:23.101279 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 12914
I20260811 10:11:23.101436 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal/instance
I20260811 10:11:23.101729 23236 external_mini_cluster.cc:1428] Setting key 31a998aaee18d4191e8e01efb62471da
I20260811 10:11:23.102896 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.177.3:0
--local_ip_for_outbound_sockets=127.22.177.3
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:44633
--builtin_ntp_servers=127.22.177.20:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:11:23.105721 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.100429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.2:36101 (local address 127.22.177.62:44633)
0811 10:11:23.100515 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.100517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.100761 (+ 244us) server_negotiation.cc:407] Connection header received
0811 10:11:23.101356 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.101358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.101371 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.101449 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:23.102355 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.102874 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.103996 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.104202 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.104287 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.105358 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:23.105360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:23.105360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:23.105365 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:23.105381 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:23.105385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:23.105438 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.105559 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.105586 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:11:23.105990 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.100392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:44633 (local address 127.22.177.2:36101)
0811 10:11:23.100610 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.100646 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.101181 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.101526 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.101530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.101541 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:23.102247 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.102253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.102993 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.102994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.103890 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.103893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.104298 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.104299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.104474 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.105113 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:23.105114 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:23.105156 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:23.105187 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:23.105460 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:23.105466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:23.105469 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.105552 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.105717 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260811 10:11:23.106679 13061 heartbeater.cc:347] Connected to a master server at 127.22.177.62:44633
I20260811 10:11:23.106817 13061 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:23.107054 13061 heartbeater.cc:511] Master 127.22.177.62:44633 requested a full tablet report, sending...
I20260811 10:11:23.107573 12697 ts_manager.cc:194] Registered new tserver with Master: f1d9344d63074fa7bd3461465379a9d4 (127.22.177.2:41819)
I20260811 10:11:23.108014 12697 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.2:36101
W20260811 10:11:23.189543 13064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:23.189777 13064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:23.189800 13064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:23.190995 13064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:23.191039 13064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:23.191066 13064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.177.3
I20260811 10:11:23.192420 13064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.177.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.177.3
--webserver_port=0
--tserver_master_addrs=127.22.177.62:44633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.177.3
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:23.192674 13064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:23.193008 13064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:23.198354 13072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:23.198377 13071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:23.198354 13074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:23.198696 13064 server_base.cc:1034] running on GCE node
I20260811 10:11:23.198971 13064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:23.199193 13064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:23.200382 13064 hybrid_clock.cc:648] HybridClock initialized: now 1786443083200368 us; error 31 us; skew 500 ppm
I20260811 10:11:23.201862 13064 webserver.cc:460] Webserver started at http://127.22.177.3:43041/ using document root <none> and password file <none>
I20260811 10:11:23.202119 13064 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:23.202281 13064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:23.202441 13064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:23.203577 13064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data/instance:
uuid: "272a508844564398a9835912eeedbd87"
format_stamp: "Formatted at 2026-08-11 10:11:23 on dist-test-slave-g350"
server_key: "59d9acffaa3a9cef7e07fdbfd56e284e"
server_key_iv: "564b69a1028752216101dd022a05bff8"
server_key_version: "encryptionkey@0"
I20260811 10:11:23.203980 13064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance:
uuid: "272a508844564398a9835912eeedbd87"
format_stamp: "Formatted at 2026-08-11 10:11:23 on dist-test-slave-g350"
server_key: "59d9acffaa3a9cef7e07fdbfd56e284e"
server_key_iv: "564b69a1028752216101dd022a05bff8"
server_key_version: "encryptionkey@0"
I20260811 10:11:23.205366 13064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:23.206394 13082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:23.206640 13064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:23.206754 13064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal
uuid: "272a508844564398a9835912eeedbd87"
format_stamp: "Formatted at 2026-08-11 10:11:23 on dist-test-slave-g350"
server_key: "59d9acffaa3a9cef7e07fdbfd56e284e"
server_key_iv: "564b69a1028752216101dd022a05bff8"
server_key_version: "encryptionkey@0"
I20260811 10:11:23.206846 13064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:23.227228 13064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:23.227682 13064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:23.227878 13064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:23.228188 13064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:11:23.228605 13064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:11:23.228674 13064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:23.228721 13064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:11:23.228754 13064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:23.235409 13064 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.3:43555
I20260811 10:11:23.235466 13210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.3:43555 every 8 connection(s)
I20260811 10:11:23.235904 13064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/data/info.pb
I20260811 10:11:23.240139 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13064
I20260811 10:11:23.240274 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal/instance
I20260811 10:11:23.240523 23236 external_mini_cluster.cc:1428] Setting key 73f386d58010b6c5542dd795ff440264
I20260811 10:11:23.242230 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.236753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:43165 (local address 127.22.177.62:44633)
0811 10:11:23.236828 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.236830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.237097 (+ 267us) server_negotiation.cc:407] Connection header received
0811 10:11:23.237699 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.237700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.237714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.237786 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:23.238562 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.239117 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.240422 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.240692 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.240773 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.241920 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:23.241922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:23.241922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:23.241928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:23.241944 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:23.241948 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:23.242028 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.242112 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.242142 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:11:23.242501 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.236738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:44633 (local address 127.22.177.3:43165)
0811 10:11:23.236929 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.236994 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.237561 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.237899 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.237902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.237911 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:23.238453 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.238458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.239227 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.239228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.240328 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.240332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.240801 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.240802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.241009 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.241742 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:11:23.241742 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:23.241774 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:11:23.241800 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:23.242030 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:23.242037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:23.242041 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.242118 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.242232 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260811 10:11:23.242998 13212 heartbeater.cc:347] Connected to a master server at 127.22.177.62:44633
I20260811 10:11:23.243084 13212 heartbeater.cc:464] Registering TS with master...
I20260811 10:11:23.243235 13212 heartbeater.cc:511] Master 127.22.177.62:44633 requested a full tablet report, sending...
I20260811 10:11:23.243587 12697 ts_manager.cc:194] Registered new tserver with Master: 272a508844564398a9835912eeedbd87 (127.22.177.3:43555)
I20260811 10:11:23.244001 12697 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.177.3:43165
I20260811 10:11:23.252985 23236 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:11:23.258257 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.254766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.22.177.62:44633)
0811 10:11:23.254814 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.254816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.254957 (+ 141us) server_negotiation.cc:407] Connection header received
0811 10:11:23.255021 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.255022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.255032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.255079 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:23.255228 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.255812 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.256683 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.256918 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.256987 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.257893 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:23.257895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:23.257895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:23.257900 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:23.257914 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:23.257917 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:23.257967 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.258148 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.258181 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:23.260182 12697 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:11:23.261952 12697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:11:23.268960 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.264809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.22.177.2:41819)
0811 10:11:23.265071 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.265076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.265088 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:23.265134 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.265136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.265272 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.265664 (+ 392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:23.265738 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.266430 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.267688 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.268525 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.268593 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.268697 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.268817 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.268860 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":98,"threads_started":1}
I20260811 10:11:23.268949 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.264554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:41819 (local address 127.0.0.1:39710)
0811 10:11:23.264697 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.264756 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.264838 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.265464 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.265467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.265469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:23.265554 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.265557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.266521 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.266523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.267607 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.267610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.268601 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.268602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.268690 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.268706 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.268797 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.268861 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260811 10:11:23.269496 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.264619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.3:43555 (local address 127.0.0.1:45182)
0811 10:11:23.264772 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.264785 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.264838 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.265481 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.265482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.265485 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:23.265575 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.265577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.267135 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.267136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.268026 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.268028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.268833 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.268834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.268903 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.268907 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.268936 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.268975 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260811 10:11:23.269965 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.264619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:33145 (local address 127.0.0.1:42932)
0811 10:11:23.265134 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.265144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.265274 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.265649 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.265650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.265652 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:23.265691 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.265693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.267157 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.267158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.268419 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.268422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.269734 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.269735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.269799 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.269802 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.269856 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.269896 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":109,"threads_started":1}
I20260811 10:11:23.269979 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.264857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.22.177.3:43555)
0811 10:11:23.265076 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.265081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.265093 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:23.265145 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.265147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.265164 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.265706 (+ 542us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:23.265793 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.267033 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.268127 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.268731 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.269721 (+ 990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.269820 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.269846 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.269896 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260811 10:11:23.270051 12988 tablet_service.cc:1467] Processing CreateTablet for tablet da08df8542664f0b94765badaba6cf4f (DEFAULT_TABLE table=test-table [id=67e1fd759d804a2f9d524490bf995cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:23.270232 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.265144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.22.177.1:33145)
0811 10:11:23.265364 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.265368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.265379 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:11:23.265423 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.265425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.265473 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.265556 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:23.265801 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.267059 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.268524 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.269480 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.269553 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.269695 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.270091 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.270143 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":95,"threads_started":1}
I20260811 10:11:23.270511 12988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da08df8542664f0b94765badaba6cf4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:23.270782 12836 tablet_service.cc:1467] Processing CreateTablet for tablet da08df8542664f0b94765badaba6cf4f (DEFAULT_TABLE table=test-table [id=67e1fd759d804a2f9d524490bf995cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:23.270763 13134 tablet_service.cc:1467] Processing CreateTablet for tablet da08df8542664f0b94765badaba6cf4f (DEFAULT_TABLE table=test-table [id=67e1fd759d804a2f9d524490bf995cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:11:23.271119 12836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da08df8542664f0b94765badaba6cf4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:23.271119 13134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da08df8542664f0b94765badaba6cf4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:23.273754 13227 tablet_bootstrap.cc:492] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Bootstrap starting.
I20260811 10:11:23.274405 13227 tablet_bootstrap.cc:654] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:23.274694 13228 tablet_bootstrap.cc:492] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Bootstrap starting.
I20260811 10:11:23.274816 13227 log.cc:824] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:23.275475 13228 tablet_bootstrap.cc:654] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:23.275662 13227 tablet_bootstrap.cc:492] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: No bootstrap required, opened a new log
I20260811 10:11:23.275763 13227 ts_tablet_manager.cc:1397] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:11:23.275983 13228 log.cc:824] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:23.276389 13231 tablet_bootstrap.cc:492] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: Bootstrap starting.
I20260811 10:11:23.277160 13231 tablet_bootstrap.cc:654] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:23.277508 13231 log.cc:824] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:23.278110 13228 tablet_bootstrap.cc:492] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: No bootstrap required, opened a new log
I20260811 10:11:23.278281 13228 ts_tablet_manager.cc:1397] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 10:11:23.278457 13231 tablet_bootstrap.cc:492] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: No bootstrap required, opened a new log
I20260811 10:11:23.278541 13231 ts_tablet_manager.cc:1397] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:11:23.279040 13227 raft_consensus.cc:348] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.279285 13227 raft_consensus.cc:374] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:23.279330 13227 raft_consensus.cc:729] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d9344d63074fa7bd3461465379a9d4, State: Initialized, Role: FOLLOWER
I20260811 10:11:23.279510 13227 consensus_queue.cc:260] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.279805 13227 ts_tablet_manager.cc:1428] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260811 10:11:23.279973 13061 heartbeater.cc:503] Master 127.22.177.62:44633 was elected leader, sending a full tablet report...
I20260811 10:11:23.281219 13228 raft_consensus.cc:348] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.281399 13228 raft_consensus.cc:374] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:23.281455 13228 raft_consensus.cc:729] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272a508844564398a9835912eeedbd87, State: Initialized, Role: FOLLOWER
I20260811 10:11:23.281628 13228 consensus_queue.cc:260] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.281715 13231 raft_consensus.cc:348] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.281886 13231 raft_consensus.cc:374] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:23.281936 13231 raft_consensus.cc:729] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c3f7074cce43b89f5c5f3c77953549, State: Initialized, Role: FOLLOWER
I20260811 10:11:23.281967 13228 ts_tablet_manager.cc:1428] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 10:11:23.282099 13212 heartbeater.cc:503] Master 127.22.177.62:44633 was elected leader, sending a full tablet report...
I20260811 10:11:23.282074 13231 consensus_queue.cc:260] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.282349 13231 ts_tablet_manager.cc:1428] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260811 10:11:23.282547 12911 heartbeater.cc:503] Master 127.22.177.62:44633 was elected leader, sending a full tablet report...
W20260811 10:11:23.350179 13062 tablet.cc:2369] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:11:23.435736 12912 tablet.cc:2369] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:11:23.437537 13236 raft_consensus.cc:482] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:11:23.437691 13236 raft_consensus.cc:504] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.438046 13236 leader_election.cc:288] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1d9344d63074fa7bd3461465379a9d4 (127.22.177.2:41819), 94c3f7074cce43b89f5c5f3c77953549 (127.22.177.1:33145)
I20260811 10:11:23.441169 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.438483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:56623 (local address 127.22.177.2:41819)
0811 10:11:23.438527 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.438530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.438541 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:11:23.438604 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.438605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.438617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.438697 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:23.438944 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.439436 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.440408 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.440779 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.440872 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.440881 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.441042 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.441082 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:23.441195 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.438518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.177.3:56647 (local address 127.22.177.1:33145)
0811 10:11:23.438562 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.438564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.438697 (+ 133us) server_negotiation.cc:407] Connection header received
0811 10:11:23.438767 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.438768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.438779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.438852 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:11:23.439033 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.439506 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.440476 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.440829 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.440916 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.440924 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.441083 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.441122 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:11:23.441203 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.438297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.2:41819 (local address 127.22.177.3:56623)
0811 10:11:23.438449 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.438464 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.438534 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.438702 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.438703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.438704 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:23.438789 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.438791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.439530 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.439531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.440293 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.440295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.440877 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.440878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.440959 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.440967 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.441035 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.441082 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 10:11:23.441219 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.438304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:33145 (local address 127.22.177.3:56647)
0811 10:11:23.438596 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.438607 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:23.438704 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:23.438875 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:23.438876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:23.438878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:11:23.438952 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.438953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.439598 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.439599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.440372 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:23.440374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.440917 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.440918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:23.440997 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.441005 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 10:11:23.441075 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:11:23.441113 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":95,"thread_start_us":111,"threads_started":1}
I20260811 10:11:23.441677 13010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da08df8542664f0b94765badaba6cf4f" candidate_uuid: "272a508844564398a9835912eeedbd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d9344d63074fa7bd3461465379a9d4" is_pre_election: true
I20260811 10:11:23.441684 12858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da08df8542664f0b94765badaba6cf4f" candidate_uuid: "272a508844564398a9835912eeedbd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94c3f7074cce43b89f5c5f3c77953549" is_pre_election: true
I20260811 10:11:23.441852 13010 raft_consensus.cc:2454] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 272a508844564398a9835912eeedbd87 in term 0.
I20260811 10:11:23.441852 12858 raft_consensus.cc:2454] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 272a508844564398a9835912eeedbd87 in term 0.
I20260811 10:11:23.442129 13083 leader_election.cc:302] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 272a508844564398a9835912eeedbd87, f1d9344d63074fa7bd3461465379a9d4; no voters:
I20260811 10:11:23.442276 13236 raft_consensus.cc:2781] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:11:23.442339 13236 raft_consensus.cc:482] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:11:23.442364 13236 raft_consensus.cc:3037] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:23.443048 13236 raft_consensus.cc:504] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.443202 13236 leader_election.cc:288] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [CANDIDATE]: Term 1 election: Requested vote from peers f1d9344d63074fa7bd3461465379a9d4 (127.22.177.2:41819), 94c3f7074cce43b89f5c5f3c77953549 (127.22.177.1:33145)
I20260811 10:11:23.443390 13010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da08df8542664f0b94765badaba6cf4f" candidate_uuid: "272a508844564398a9835912eeedbd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d9344d63074fa7bd3461465379a9d4"
I20260811 10:11:23.443390 12858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da08df8542664f0b94765badaba6cf4f" candidate_uuid: "272a508844564398a9835912eeedbd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94c3f7074cce43b89f5c5f3c77953549"
I20260811 10:11:23.443482 12858 raft_consensus.cc:3037] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:23.443481 13010 raft_consensus.cc:3037] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:23.444415 12858 raft_consensus.cc:2454] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 272a508844564398a9835912eeedbd87 in term 1.
I20260811 10:11:23.444415 13010 raft_consensus.cc:2454] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 272a508844564398a9835912eeedbd87 in term 1.
I20260811 10:11:23.444645 13084 leader_election.cc:302] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 272a508844564398a9835912eeedbd87, 94c3f7074cce43b89f5c5f3c77953549; no voters:
I20260811 10:11:23.444806 13236 raft_consensus.cc:2781] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:23.444986 13236 raft_consensus.cc:686] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 1 LEADER]: Becoming Leader. State: Replica: 272a508844564398a9835912eeedbd87, State: Running, Role: LEADER
I20260811 10:11:23.445096 13236 consensus_queue.cc:237] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } }
I20260811 10:11:23.446028 12697 catalog_manager.cc:5495] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 272a508844564398a9835912eeedbd87 (127.22.177.3). New cstate: current_term: 1 leader_uuid: "272a508844564398a9835912eeedbd87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "272a508844564398a9835912eeedbd87" member_type: VOTER last_known_addr { host: "127.22.177.3" port: 43555 } health_report { overall_health: HEALTHY } } }
I20260811 10:11:23.452188 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.449403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45194 (local address 127.22.177.3:43555)
0811 10:11:23.449488 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:23.449490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:23.449498 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:23.449528 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:23.449529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:23.449544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:23.449669 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:23.449885 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.450326 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.451135 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:23.451335 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:23.451429 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:23.451646 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:23.451738 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:23.451843 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:23.452049 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 10:11:23.452066 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 10:11:23.454505 13010 raft_consensus.cc:1261] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 1 FOLLOWER]: Refusing update from remote peer 272a508844564398a9835912eeedbd87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:23.454505 12858 raft_consensus.cc:1261] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 1 FOLLOWER]: Refusing update from remote peer 272a508844564398a9835912eeedbd87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:23.454912 13236 consensus_queue.cc:1035] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1d9344d63074fa7bd3461465379a9d4" member_type: VOTER last_known_addr { host: "127.22.177.2" port: 41819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:23.455083 13240 consensus_queue.cc:1035] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94c3f7074cce43b89f5c5f3c77953549" member_type: VOTER last_known_addr { host: "127.22.177.1" port: 33145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:23.457573 13243 mvcc.cc:205] Tried to move back new op lower bound from 7317270869826441216 to 7317270869791768576. Current Snapshot: MvccSnapshot[applied={T|T < 7317270869826441216}]
I20260811 10:11:23.457926 13244 mvcc.cc:205] Tried to move back new op lower bound from 7317270869826441216 to 7317270869791768576. Current Snapshot: MvccSnapshot[applied={T|T < 7317270869826441216}]
I20260811 10:11:23.460098 12696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:23.460206 12696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:11:23.461740 12696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d82746217a7d46ccba7bc355325d2e30: Sending DeleteTablet for 3 replicas of tablet da08df8542664f0b94765badaba6cf4f
I20260811 10:11:23.462116 12988 tablet_service.cc:1514] Processing DeleteTablet for tablet da08df8542664f0b94765badaba6cf4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:23 UTC) from {username='slave'} at 127.0.0.1:39710
I20260811 10:11:23.462141 13134 tablet_service.cc:1514] Processing DeleteTablet for tablet da08df8542664f0b94765badaba6cf4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:23 UTC) from {username='slave'} at 127.0.0.1:45182
I20260811 10:11:23.462198 12836 tablet_service.cc:1514] Processing DeleteTablet for tablet da08df8542664f0b94765badaba6cf4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:11:23 UTC) from {username='slave'} at 127.0.0.1:42932
I20260811 10:11:23.462457 13257 tablet_replica.cc:330] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549: stopping tablet replica
I20260811 10:11:23.462507 13255 tablet_replica.cc:330] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: stopping tablet replica
I20260811 10:11:23.462630 13255 raft_consensus.cc:2229] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:23.462647 13257 raft_consensus.cc:2229] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:11:23.462716 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12769
I20260811 10:11:23.462752 13255 raft_consensus.cc:2258] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:23.462914 13257 raft_consensus.cc:2258] T da08df8542664f0b94765badaba6cf4f P 94c3f7074cce43b89f5c5f3c77953549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:23.467370 13255 ts_tablet_manager.cc:1905] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:23.469758 13255 ts_tablet_manager.cc:1918] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:23.469861 13255 log.cc:1196] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-1/wal/wals/da08df8542664f0b94765badaba6cf4f
I20260811 10:11:23.470166 13255 ts_tablet_manager.cc:1939] T da08df8542664f0b94765badaba6cf4f P f1d9344d63074fa7bd3461465379a9d4: Deleting consensus metadata
I20260811 10:11:23.470727 12678 catalog_manager.cc:4841] TS f1d9344d63074fa7bd3461465379a9d4 (127.22.177.2:41819): tablet da08df8542664f0b94765badaba6cf4f (table test-table [id=67e1fd759d804a2f9d524490bf995cc5]) successfully deleted
W20260811 10:11:23.471912 12679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.177.1:33145 (error 108)
I20260811 10:11:23.472105 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12914
I20260811 10:11:23.473701 13256 tablet_replica.cc:330] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: stopping tablet replica
I20260811 10:11:23.473826 13256 raft_consensus.cc:2229] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:11:23.473937 13256 raft_consensus.cc:2258] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:11:23.479633 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:23.473275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.1:33145 (local address 127.0.0.1:42942)
0811 10:11:23.479499 (+ 6224us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:23.479585 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.1:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6196}
W20260811 10:11:23.479825 12679 catalog_manager.cc:4584] TS 94c3f7074cce43b89f5c5f3c77953549 (127.22.177.1:33145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da08df8542664f0b94765badaba6cf4f: Network error: Client connection negotiation failed: client connection to 127.22.177.1:33145: connect: Connection refused (error 111)
I20260811 10:11:23.480413 13256 ts_tablet_manager.cc:1905] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:11:23.482124 13256 ts_tablet_manager.cc:1918] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:11:23.482193 13256 log.cc:1196] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Deleting WAL directory at /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786442825609099-23236-0/minicluster-data/ts-2/wal/wals/da08df8542664f0b94765badaba6cf4f
I20260811 10:11:23.482443 13256 ts_tablet_manager.cc:1939] T da08df8542664f0b94765badaba6cf4f P 272a508844564398a9835912eeedbd87: Deleting consensus metadata
I20260811 10:11:23.482843 12680 catalog_manager.cc:4841] TS 272a508844564398a9835912eeedbd87 (127.22.177.3:43555): tablet da08df8542664f0b94765badaba6cf4f (table test-table [id=67e1fd759d804a2f9d524490bf995cc5]) successfully deleted
I20260811 10:11:23.486202 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13064
I20260811 10:11:23.494701 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 12660
2026-08-11T10:11:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (882 ms)
[----------] 6 tests from EncryptionPolicyTest (8917 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:23 dist-test-slave-g350 krb5kdc[13262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:23 dist-test-slave-g350 krb5kdc[13262](info): set up 2 sockets
Aug 11 10:11:23 dist-test-slave-g350 krb5kdc[13262](info): commencing operation
krb5kdc: starting...
W20260811 10:11:24.657191 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:24 dist-test-slave-g350 krb5kdc[13262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:24.788837 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:24.874840 13277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:24.875115 13277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:24.875170 13277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:24.876416 13277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:24.876492 13277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:24.876518 13277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:24.876538 13277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:24.876569 13277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:24.878206 13277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:24.878506 13277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:24.878744 13277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:24.884122 13282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:24.884171 13277 server_base.cc:1034] running on GCE node
W20260811 10:11:24.884179 13286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:24.884133 13283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:24.884672 13277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:24.884996 13277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:24.886176 13277 hybrid_clock.cc:648] HybridClock initialized: now 1786443084886145 us; error 64 us; skew 500 ppm
Aug 11 10:11:24 dist-test-slave-g350 krb5kdc[13262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443084, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:24.894066 13277 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:24.894646 13277 webserver.cc:460] Webserver started at http://127.0.0.1:45637/ using document root <none> and password file <none>
I20260811 10:11:24.894876 13277 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:24.894922 13277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:24.895016 13277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:24.896072 13277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "4fa78d32a6fd48d19753e391c8a0a272"
format_stamp: "Formatted at 2026-08-11 10:11:24 on dist-test-slave-g350"
server_key: "9a0f2be0734ba9c064fa241885336cf9"
server_key_iv: "a796d127bce9ec5df5f11ff5951eb393"
server_key_version: "encryptionkey@0"
I20260811 10:11:24.896370 13277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "4fa78d32a6fd48d19753e391c8a0a272"
format_stamp: "Formatted at 2026-08-11 10:11:24 on dist-test-slave-g350"
server_key: "9a0f2be0734ba9c064fa241885336cf9"
server_key_iv: "a796d127bce9ec5df5f11ff5951eb393"
server_key_version: "encryptionkey@0"
I20260811 10:11:24.897948 13277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:11:24.898725 13294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:24.898897 13277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:24.898998 13277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "4fa78d32a6fd48d19753e391c8a0a272"
format_stamp: "Formatted at 2026-08-11 10:11:24 on dist-test-slave-g350"
server_key: "9a0f2be0734ba9c064fa241885336cf9"
server_key_iv: "a796d127bce9ec5df5f11ff5951eb393"
server_key_version: "encryptionkey@0"
I20260811 10:11:24.899053 13277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:24.921416 13277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:24.923717 13277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:24.923861 13277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:24.928011 13277 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43761
I20260811 10:11:24.928079 13352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43761 every 8 connection(s)
I20260811 10:11:24.928426 13277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:24.929381 13353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:24.931707 13353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272: Bootstrap starting.
I20260811 10:11:24.932379 13353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:24.932689 13353 log.cc:824] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:24.933480 13353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272: No bootstrap required, opened a new log
I20260811 10:11:24.934950 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13277
I20260811 10:11:24.935057 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:24.935199 13353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa78d32a6fd48d19753e391c8a0a272" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260811 10:11:24.935312 23236 external_mini_cluster.cc:1428] Setting key b02501ca596183ea4ed00e32af1946d3
I20260811 10:11:24.935351 13353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:24.935400 13353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa78d32a6fd48d19753e391c8a0a272, State: Initialized, Role: FOLLOWER
I20260811 10:11:24.935557 13353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa78d32a6fd48d19753e391c8a0a272" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260811 10:11:24.935649 13353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:24.935695 13353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:24.935760 13353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:24.936833 13353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa78d32a6fd48d19753e391c8a0a272" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260811 10:11:24.936924 13353 leader_election.cc:302] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fa78d32a6fd48d19753e391c8a0a272; no voters:
I20260811 10:11:24.937095 13353 leader_election.cc:288] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:24.937124 13359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:24.937243 13359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [term 1 LEADER]: Becoming Leader. State: Replica: 4fa78d32a6fd48d19753e391c8a0a272, State: Running, Role: LEADER
I20260811 10:11:24.937328 13359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa78d32a6fd48d19753e391c8a0a272" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260811 10:11:24.937366 13353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:24.937686 13359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fa78d32a6fd48d19753e391c8a0a272. Latest consensus state: current_term: 1 leader_uuid: "4fa78d32a6fd48d19753e391c8a0a272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa78d32a6fd48d19753e391c8a0a272" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } } }
I20260811 10:11:24.937762 13359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:24.937724 13360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fa78d32a6fd48d19753e391c8a0a272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa78d32a6fd48d19753e391c8a0a272" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } } }
I20260811 10:11:24.937975 13360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:24.937997 13364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:24.938546 13364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:24.940305 13364 catalog_manager.cc:1347] Generated new cluster ID: 2686730ee00244a7a93f93456a6631f3
I20260811 10:11:24.940357 13364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:11:24 dist-test-slave-g350 krb5kdc[13262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:11:24.947245 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:24.935855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47625 (local address 127.0.0.1:43761)
0811 10:11:24.936086 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:24.936096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:24.936135 (+ 39us) server_negotiation.cc:407] Connection header received
0811 10:11:24.936262 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:24.936272 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:24.936328 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:24.936501 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:24.937366 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:24.938786 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:24.939824 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:24.940430 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:24.940532 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:24.943328 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:24.943332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:24.943332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:24.943345 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:24.945407 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:24.945930 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:24.945934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:24.945935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:24.946067 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:24.946303 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:24.946304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:24.946305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:24.946623 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:24.946823 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:24.946906 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:11:24.947034 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":115,"threads_started":1}
I20260811 10:11:24.948436 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:24.957628 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:24.949969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51721 (local address 127.0.0.1:43761)
0811 10:11:24.950030 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:24.950032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:24.950251 (+ 219us) server_negotiation.cc:407] Connection header received
0811 10:11:24.950298 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:24.950299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:24.950309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:24.950379 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:24.950976 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:24.951790 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:24.952775 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:24.953115 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:24.953188 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:24.954646 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:24.954647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:24.954648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:24.954663 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:24.956314 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:24.956749 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:24.956751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:24.956751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:24.956814 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:24.957040 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:24.957041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:24.957041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:24.957173 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:24.957290 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:24.957424 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 10:11:24.957493 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:11:24.958571 13364 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:24.959055 13364 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:24.965466 13364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fa78d32a6fd48d19753e391c8a0a272: Generated new TSK 0
I20260811 10:11:24.965729 13364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:24.971669 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13277
W20260811 10:11:24.978796 13260 connection.cc:535] client connection to 127.0.0.1:43761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43761): Connection reset by peer (error 104)
2026-08-11T10:11:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:25 dist-test-slave-g350 krb5kdc[13388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:25 dist-test-slave-g350 krb5kdc[13388](info): set up 2 sockets
Aug 11 10:11:25 dist-test-slave-g350 krb5kdc[13388](info): commencing operation
krb5kdc: starting...
W20260811 10:11:26.083036 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:26 dist-test-slave-g350 krb5kdc[13388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:26.217867 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:26.307256 13409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:26.307534 13409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:26.307576 13409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:26.308796 13409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:26.308876 13409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:26.308900 13409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:26.308928 13409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:26.308957 13409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:26.310559 13409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33045
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:26.310869 13409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:26.311113 13409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:26.316480 13417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:26.316500 13414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:26.316501 13409 server_base.cc:1034] running on GCE node
W20260811 10:11:26.316633 13415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:26.317099 13409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:26.317420 13409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:26.318580 13409 hybrid_clock.cc:648] HybridClock initialized: now 1786443086318583 us; error 60 us; skew 500 ppm
Aug 11 10:11:26 dist-test-slave-g350 krb5kdc[13388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:26.321437 13409 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:26.321985 13409 webserver.cc:460] Webserver started at http://127.0.0.1:45643/ using document root <none> and password file <none>
I20260811 10:11:26.322245 13409 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:26.322294 13409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:26.322424 13409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:26.323520 13409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "aeb6b3ad558a430e882eb732b432d8cc"
format_stamp: "Formatted at 2026-08-11 10:11:26 on dist-test-slave-g350"
server_key: "bb7184f9cabd192d268dcdfc836c0c2d"
server_key_iv: "21ff9af1187e314c70b3f13f67745f46"
server_key_version: "encryptionkey@0"
I20260811 10:11:26.323885 13409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "aeb6b3ad558a430e882eb732b432d8cc"
format_stamp: "Formatted at 2026-08-11 10:11:26 on dist-test-slave-g350"
server_key: "bb7184f9cabd192d268dcdfc836c0c2d"
server_key_iv: "21ff9af1187e314c70b3f13f67745f46"
server_key_version: "encryptionkey@0"
I20260811 10:11:26.325268 13409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:26.326166 13426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:26.326356 13409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:11:26.326454 13409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "aeb6b3ad558a430e882eb732b432d8cc"
format_stamp: "Formatted at 2026-08-11 10:11:26 on dist-test-slave-g350"
server_key: "bb7184f9cabd192d268dcdfc836c0c2d"
server_key_iv: "21ff9af1187e314c70b3f13f67745f46"
server_key_version: "encryptionkey@0"
I20260811 10:11:26.326527 13409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:26.345112 13409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:26.347719 13409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:26.347890 13409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:26.352501 13409 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33045
I20260811 10:11:26.352561 13480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33045 every 8 connection(s)
I20260811 10:11:26.352936 13409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:26.353921 13481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:26.354636 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13409
I20260811 10:11:26.354768 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:26.354967 23236 external_mini_cluster.cc:1428] Setting key 915baed3e09733070ca7e7d6a9462607
I20260811 10:11:26.357005 13481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc: Bootstrap starting.
I20260811 10:11:26.357673 13481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:26.358018 13481 log.cc:824] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:26.358853 13481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc: No bootstrap required, opened a new log
I20260811 10:11:26.360849 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:26.355559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40243 (local address 127.0.0.1:33045)
0811 10:11:26.355764 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:26.355771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:26.355797 (+ 26us) server_negotiation.cc:407] Connection header received
0811 10:11:26.356204 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:26.356210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:26.356255 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:26.356371 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:26.356654 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:26.357744 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:26.358775 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:26.359254 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:26.359342 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:26.360137 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:26.360141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:26.360141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:26.360315 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:26.360346 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:26.360365 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:26.360440 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:26.360558 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 10:11:26.360608 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":115,"threads_started":1}
I20260811 10:11:26.361274 13481 raft_consensus.cc:348] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb6b3ad558a430e882eb732b432d8cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33045 } }
I20260811 10:11:26.361435 13481 raft_consensus.cc:374] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:26.361472 13481 raft_consensus.cc:729] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb6b3ad558a430e882eb732b432d8cc, State: Initialized, Role: FOLLOWER
I20260811 10:11:26.361588 13481 consensus_queue.cc:260] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb6b3ad558a430e882eb732b432d8cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33045 } }
I20260811 10:11:26.361754 13481 raft_consensus.cc:388] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:26.361791 13481 raft_consensus.cc:482] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:26.361853 13481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:26.361896 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:26.362452 13481 raft_consensus.cc:504] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb6b3ad558a430e882eb732b432d8cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33045 } }
I20260811 10:11:26.362569 13481 leader_election.cc:302] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aeb6b3ad558a430e882eb732b432d8cc; no voters:
I20260811 10:11:26.362753 13481 leader_election.cc:288] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:26.362785 13488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:26.362912 13488 raft_consensus.cc:686] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [term 1 LEADER]: Becoming Leader. State: Replica: aeb6b3ad558a430e882eb732b432d8cc, State: Running, Role: LEADER
I20260811 10:11:26.363027 13481 sys_catalog.cc:564] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:26.363051 13488 consensus_queue.cc:237] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb6b3ad558a430e882eb732b432d8cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33045 } }
I20260811 10:11:26.363484 13492 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aeb6b3ad558a430e882eb732b432d8cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb6b3ad558a430e882eb732b432d8cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33045 } } }
I20260811 10:11:26.363484 13493 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeb6b3ad558a430e882eb732b432d8cc. Latest consensus state: current_term: 1 leader_uuid: "aeb6b3ad558a430e882eb732b432d8cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb6b3ad558a430e882eb732b432d8cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33045 } } }
I20260811 10:11:26.363600 13492 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:26.363606 13493 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:26.363873 13498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:26.364461 13498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:26.366154 13498 catalog_manager.cc:1347] Generated new cluster ID: 02a1c1758e2f4783a5f92a328ee6515f
I20260811 10:11:26.366210 13498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:26.367336 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:26.363465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45801 (local address 127.0.0.1:33045)
0811 10:11:26.363499 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:26.363501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:26.363854 (+ 353us) server_negotiation.cc:407] Connection header received
0811 10:11:26.363969 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:26.363970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:26.363981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:26.364011 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:26.364233 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:26.365001 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:26.365973 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:26.366304 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:26.366396 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:26.367088 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:26.367090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:26.367091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:26.367096 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:26.367110 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:26.367115 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:26.367169 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:26.367232 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 10:11:26.367241 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 10:11:26.375771 13498 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:26.376382 13498 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:26.383520 13498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aeb6b3ad558a430e882eb732b432d8cc: Generated new TSK 0
I20260811 10:11:26.383734 13498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:26.389755 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13409
W20260811 10:11:26.397051 13386 connection.cc:535] client connection to 127.0.0.1:33045 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33045): Connection reset by peer (error 104)
2026-08-11T10:11:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:26 dist-test-slave-g350 krb5kdc[13523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:26 dist-test-slave-g350 krb5kdc[13523](info): set up 2 sockets
Aug 11 10:11:26 dist-test-slave-g350 krb5kdc[13523](info): commencing operation
krb5kdc: starting...
W20260811 10:11:27.497826 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:27 dist-test-slave-g350 krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:27.637127 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:27.722638 13537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:27.722900 13537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:27.722929 13537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:27.724136 13537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:27.724184 13537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:27.724198 13537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:27.724211 13537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:27.724223 13537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:27.725827 13537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:27.726094 13537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:27.726342 13537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:27.731528 13545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:27.731528 13549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:27.731601 13547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:27.731797 13537 server_base.cc:1034] running on GCE node
I20260811 10:11:27.732203 13537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:27.732599 13537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:27.733769 13537 hybrid_clock.cc:648] HybridClock initialized: now 1786443087733733 us; error 45 us; skew 500 ppm
Aug 11 10:11:27 dist-test-slave-g350 krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:27.736626 13537 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:27.737138 13537 webserver.cc:460] Webserver started at http://127.0.0.1:41337/ using document root <none> and password file <none>
I20260811 10:11:27.737402 13537 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:27.737471 13537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:27.737655 13537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:27.738790 13537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "2b74ceb782e34b2db897818cb03e1aec"
format_stamp: "Formatted at 2026-08-11 10:11:27 on dist-test-slave-g350"
server_key: "ee91f0920f03e922eb194a8e13d67f33"
server_key_iv: "58b7394ca708be994fa62622aa1facf1"
server_key_version: "encryptionkey@0"
I20260811 10:11:27.739192 13537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "2b74ceb782e34b2db897818cb03e1aec"
format_stamp: "Formatted at 2026-08-11 10:11:27 on dist-test-slave-g350"
server_key: "ee91f0920f03e922eb194a8e13d67f33"
server_key_iv: "58b7394ca708be994fa62622aa1facf1"
server_key_version: "encryptionkey@0"
I20260811 10:11:27.740638 13537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:27.741554 13556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:27.741868 13537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:27.741986 13537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "2b74ceb782e34b2db897818cb03e1aec"
format_stamp: "Formatted at 2026-08-11 10:11:27 on dist-test-slave-g350"
server_key: "ee91f0920f03e922eb194a8e13d67f33"
server_key_iv: "58b7394ca708be994fa62622aa1facf1"
server_key_version: "encryptionkey@0"
I20260811 10:11:27.742069 13537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:27.745520 13537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:27.747944 13537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:27.748122 13537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:27.752920 13537 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44369
I20260811 10:11:27.752954 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44369 every 8 connection(s)
I20260811 10:11:27.753351 13537 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:27.754534 13618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:27.756848 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec: Bootstrap starting.
I20260811 10:11:27.757405 13618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:27.757714 13618 log.cc:824] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:27.758539 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec: No bootstrap required, opened a new log
I20260811 10:11:27.760327 13618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b74ceb782e34b2db897818cb03e1aec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44369 } }
I20260811 10:11:27.760490 13618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:27.760517 13618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b74ceb782e34b2db897818cb03e1aec, State: Initialized, Role: FOLLOWER
I20260811 10:11:27.760634 13618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b74ceb782e34b2db897818cb03e1aec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44369 } }
I20260811 10:11:27.760695 13618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:27.760731 13618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:27.760762 13618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:27.761343 13618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b74ceb782e34b2db897818cb03e1aec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44369 } }
I20260811 10:11:27.761444 13618 leader_election.cc:302] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b74ceb782e34b2db897818cb03e1aec; no voters:
I20260811 10:11:27.761656 13618 leader_election.cc:288] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:27.761708 13622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:27.761891 13622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [term 1 LEADER]: Becoming Leader. State: Replica: 2b74ceb782e34b2db897818cb03e1aec, State: Running, Role: LEADER
I20260811 10:11:27.761981 13618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:27.762044 13622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b74ceb782e34b2db897818cb03e1aec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44369 } }
I20260811 10:11:27.762722 13624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b74ceb782e34b2db897818cb03e1aec. Latest consensus state: current_term: 1 leader_uuid: "2b74ceb782e34b2db897818cb03e1aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b74ceb782e34b2db897818cb03e1aec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44369 } } }
I20260811 10:11:27.762828 13624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:27.763046 13623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b74ceb782e34b2db897818cb03e1aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b74ceb782e34b2db897818cb03e1aec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44369 } } }
I20260811 10:11:27.763146 13631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:27.763434 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13537
I20260811 10:11:27.763584 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:27.763718 13631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:27.763829 23236 external_mini_cluster.cc:1428] Setting key c4bbdab82529c308c13360a439fc5519
I20260811 10:11:27.763162 13623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:27.765684 13631 catalog_manager.cc:1347] Generated new cluster ID: a7fdc31bd4474821a4ea4ee365b465bf
I20260811 10:11:27.765738 13631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:27.770462 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:27.764489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39447 (local address 127.0.0.1:44369)
0811 10:11:27.764681 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:27.764691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:27.764777 (+ 86us) server_negotiation.cc:407] Connection header received
0811 10:11:27.765078 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:27.765085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:27.765116 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:27.765282 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:27.769863 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:27.769866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:27.769866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:27.769870 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:27.769900 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:27.769912 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:27.770010 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:27.770120 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 10:11:27.770291 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260811 10:11:27.771520 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:27.778949 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:27.773055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56485 (local address 127.0.0.1:44369)
0811 10:11:27.773096 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:27.773099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:27.777829 (+ 4730us) server_negotiation.cc:407] Connection header received
0811 10:11:27.777911 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:27.777912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:27.777919 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:27.777988 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:27.778679 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:27.778680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:27.778681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:27.778685 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:27.778702 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:27.778706 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:27.778773 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:27.778837 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 10:11:27.778852 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:11:27.782929 13631 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:27.783228 13631 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:27.786480 13631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b74ceb782e34b2db897818cb03e1aec: Generated new TSK 0
I20260811 10:11:27.786671 13631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:27.791934 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13537
2026-08-11T10:11:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:27 dist-test-slave-g350 krb5kdc[13650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:27 dist-test-slave-g350 krb5kdc[13650](info): set up 2 sockets
Aug 11 10:11:27 dist-test-slave-g350 krb5kdc[13650](info): commencing operation
krb5kdc: starting...
W20260811 10:11:28.973636 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:29 dist-test-slave-g350 krb5kdc[13650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:29.113546 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:29.200990 13664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:29.201261 13664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:29.201318 13664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:29.202677 13664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:29.202749 13664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:29.202772 13664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:29.202792 13664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:29.202821 13664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:29.204386 13664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:29.204675 13664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:29.204918 13664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:29.210749 13672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:29.210740 13669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:29.210734 13670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:29.213651 13664 server_base.cc:1034] running on GCE node
I20260811 10:11:29.213981 13664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:29.214308 13664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:29.215694 13664 hybrid_clock.cc:648] HybridClock initialized: now 1786443089215677 us; error 40 us; skew 500 ppm
Aug 11 10:11:29 dist-test-slave-g350 krb5kdc[13650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443089, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:29.218880 13664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:29.219501 13664 webserver.cc:460] Webserver started at http://127.0.0.1:46083/ using document root <none> and password file <none>
I20260811 10:11:29.219769 13664 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:29.219820 13664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:29.219959 13664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:29.221105 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "713c99a9486743748120ef7a369850a8"
format_stamp: "Formatted at 2026-08-11 10:11:29 on dist-test-slave-g350"
server_key: "c5382cf69c2780a5b32e7e32eda73c2e"
server_key_iv: "9abd016c8a6bcc5f039a442a798170de"
server_key_version: "encryptionkey@0"
I20260811 10:11:29.221484 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "713c99a9486743748120ef7a369850a8"
format_stamp: "Formatted at 2026-08-11 10:11:29 on dist-test-slave-g350"
server_key: "c5382cf69c2780a5b32e7e32eda73c2e"
server_key_iv: "9abd016c8a6bcc5f039a442a798170de"
server_key_version: "encryptionkey@0"
I20260811 10:11:29.222922 13664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:29.223829 13680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:29.224016 13664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:11:29.224126 13664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "713c99a9486743748120ef7a369850a8"
format_stamp: "Formatted at 2026-08-11 10:11:29 on dist-test-slave-g350"
server_key: "c5382cf69c2780a5b32e7e32eda73c2e"
server_key_iv: "9abd016c8a6bcc5f039a442a798170de"
server_key_version: "encryptionkey@0"
I20260811 10:11:29.224200 13664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:29.234485 13664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:29.236675 13664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:29.236850 13664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:29.241415 13664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46671
I20260811 10:11:29.241456 13743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46671 every 8 connection(s)
I20260811 10:11:29.241909 13664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:29.242849 13744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:29.245335 13744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8: Bootstrap starting.
I20260811 10:11:29.246131 13744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:29.246508 13744 log.cc:824] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:29.247586 13744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8: No bootstrap required, opened a new log
I20260811 10:11:29.249861 13744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713c99a9486743748120ef7a369850a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260811 10:11:29.250028 13744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:29.250075 13744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713c99a9486743748120ef7a369850a8, State: Initialized, Role: FOLLOWER
I20260811 10:11:29.250252 13744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713c99a9486743748120ef7a369850a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260811 10:11:29.250291 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13664
I20260811 10:11:29.250334 13744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:29.250380 13744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:29.250408 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:29.250432 13744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:29.250600 23236 external_mini_cluster.cc:1428] Setting key ef1206dcb60daa8f99045418c78d1604
I20260811 10:11:29.251108 13744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713c99a9486743748120ef7a369850a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260811 10:11:29.251261 13744 leader_election.cc:302] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 713c99a9486743748120ef7a369850a8; no voters:
I20260811 10:11:29.251417 13744 leader_election.cc:288] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:29.251477 13749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:29.251654 13749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [term 1 LEADER]: Becoming Leader. State: Replica: 713c99a9486743748120ef7a369850a8, State: Running, Role: LEADER
I20260811 10:11:29.251706 13744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:29.251781 13749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713c99a9486743748120ef7a369850a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260811 10:11:29.252127 13749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 713c99a9486743748120ef7a369850a8. Latest consensus state: current_term: 1 leader_uuid: "713c99a9486743748120ef7a369850a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713c99a9486743748120ef7a369850a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } } }
I20260811 10:11:29.252215 13749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:29.252453 13754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:29.253101 13754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:29.253557 13750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "713c99a9486743748120ef7a369850a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713c99a9486743748120ef7a369850a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } } }
I20260811 10:11:29.253667 13750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:29.255599 13754 catalog_manager.cc:1347] Generated new cluster ID: 80328b4717eb4521a8bb48545128460c
I20260811 10:11:29.255673 13754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:11:29 dist-test-slave-g350 krb5kdc[13650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:11:29.262887 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:29.251150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52559 (local address 127.0.0.1:46671)
0811 10:11:29.251441 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:29.251452 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:29.251582 (+ 130us) server_negotiation.cc:407] Connection header received
0811 10:11:29.251810 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:29.251817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:29.251887 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:29.252063 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:29.253052 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:29.254470 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:29.255552 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:29.256108 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:29.256323 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:29.258968 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:29.258972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:29.258972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:29.258985 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:29.260864 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:29.261479 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:29.261483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:29.261483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:29.261675 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:29.261925 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:29.261926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:29.261927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:29.262204 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:29.262399 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:29.262474 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 10:11:29.262633 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":92,"threads_started":1}
I20260811 10:11:29.264003 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:29.270138 13754 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:29.270951 13754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:29.272631 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:29.265500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57981 (local address 127.0.0.1:46671)
0811 10:11:29.265545 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:29.265548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:29.265679 (+ 131us) server_negotiation.cc:407] Connection header received
0811 10:11:29.265760 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:29.265761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:29.265773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:29.265824 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:29.266404 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:29.266926 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:29.267675 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:29.267910 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:29.267989 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:29.269326 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:29.269328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:29.269328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:29.269336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:29.271129 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:29.271543 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:29.271544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:29.271545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:29.271598 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:29.271764 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:29.271765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:29.271765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:29.271888 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:29.271980 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:29.272440 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 10:11:29.272539 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:11:29.274245 13754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 713c99a9486743748120ef7a369850a8: Generated new TSK 0
I20260811 10:11:29.274504 13754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:29.282552 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13664
2026-08-11T10:11:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:29 dist-test-slave-g350 krb5kdc[13776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:29 dist-test-slave-g350 krb5kdc[13776](info): set up 2 sockets
Aug 11 10:11:29 dist-test-slave-g350 krb5kdc[13776](info): commencing operation
krb5kdc: starting...
W20260811 10:11:30.408174 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:30 dist-test-slave-g350 krb5kdc[13776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:30.541436 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:30.628835 13792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:30.629102 13792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:30.629143 13792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:30.630415 13792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:30.630488 13792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:30.630512 13792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:30.630533 13792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:30.630563 13792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:30.632145 13792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:30.632441 13792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:30.632683 13792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:30.637786 13801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:30.637797 13798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:30.637800 13799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:30.638381 13792 server_base.cc:1034] running on GCE node
I20260811 10:11:30.638666 13792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:30.638942 13792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:30.640116 13792 hybrid_clock.cc:648] HybridClock initialized: now 1786443090640108 us; error 36 us; skew 500 ppm
Aug 11 10:11:30 dist-test-slave-g350 krb5kdc[13776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443090, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:30.647854 13792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:30.648346 13792 webserver.cc:460] Webserver started at http://127.0.0.1:35483/ using document root <none> and password file <none>
I20260811 10:11:30.648586 13792 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:30.648654 13792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:30.648814 13792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:30.650013 13792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "1c7d93d99ed24e149788545b4454d9c0"
format_stamp: "Formatted at 2026-08-11 10:11:30 on dist-test-slave-g350"
server_key: "ddeb1eea919b27e0912a4c3613ed1bf4"
server_key_iv: "17d34bfacebc3d62adabbc0fcebe3452"
server_key_version: "encryptionkey@0"
I20260811 10:11:30.650405 13792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "1c7d93d99ed24e149788545b4454d9c0"
format_stamp: "Formatted at 2026-08-11 10:11:30 on dist-test-slave-g350"
server_key: "ddeb1eea919b27e0912a4c3613ed1bf4"
server_key_iv: "17d34bfacebc3d62adabbc0fcebe3452"
server_key_version: "encryptionkey@0"
I20260811 10:11:30.651973 13792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:30.652908 13808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:30.653137 13792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:30.653252 13792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "1c7d93d99ed24e149788545b4454d9c0"
format_stamp: "Formatted at 2026-08-11 10:11:30 on dist-test-slave-g350"
server_key: "ddeb1eea919b27e0912a4c3613ed1bf4"
server_key_iv: "17d34bfacebc3d62adabbc0fcebe3452"
server_key_version: "encryptionkey@0"
I20260811 10:11:30.653383 13792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:30.663851 13792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:30.665934 13792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:30.666098 13792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:30.670140 13792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45559
I20260811 10:11:30.670176 13871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45559 every 8 connection(s)
I20260811 10:11:30.670576 13792 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:30.671530 13872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:30.674012 13872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0: Bootstrap starting.
I20260811 10:11:30.674633 13872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:30.674924 13872 log.cc:824] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:30.675657 13872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0: No bootstrap required, opened a new log
I20260811 10:11:30.677273 13872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d93d99ed24e149788545b4454d9c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260811 10:11:30.677455 13872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:30.677481 13872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7d93d99ed24e149788545b4454d9c0, State: Initialized, Role: FOLLOWER
I20260811 10:11:30.677696 13872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d93d99ed24e149788545b4454d9c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260811 10:11:30.677809 13872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:30.677850 13872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:30.677896 13872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:30.677963 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13792
I20260811 10:11:30.678086 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:30.678274 23236 external_mini_cluster.cc:1428] Setting key f7c134c0bbb10dcabb00661c39c731de
I20260811 10:11:30.678668 13872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d93d99ed24e149788545b4454d9c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260811 10:11:30.678787 13872 leader_election.cc:302] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c7d93d99ed24e149788545b4454d9c0; no voters:
I20260811 10:11:30.678961 13872 leader_election.cc:288] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:30.679011 13880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:30.679289 13880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7d93d99ed24e149788545b4454d9c0, State: Running, Role: LEADER
I20260811 10:11:30.679425 13872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:30.679425 13880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d93d99ed24e149788545b4454d9c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260811 10:11:30.679944 13881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c7d93d99ed24e149788545b4454d9c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d93d99ed24e149788545b4454d9c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } } }
I20260811 10:11:30.679957 13883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7d93d99ed24e149788545b4454d9c0. Latest consensus state: current_term: 1 leader_uuid: "1c7d93d99ed24e149788545b4454d9c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d93d99ed24e149788545b4454d9c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } } }
I20260811 10:11:30.680045 13881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:30.680058 13883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:30.680255 13885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:30.680804 13885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:30.684751 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:30.678806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37515 (local address 127.0.0.1:45559)
0811 10:11:30.678996 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:30.679006 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:30.679046 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:11:30.679186 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:30.679194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:30.679270 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:30.679685 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:30.679798 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:30.681330 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:30.682331 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:30.682966 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:30.683093 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:30.683866 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:30.683870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:30.683870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:30.684092 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:30.684132 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:30.684148 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:30.684221 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:30.684436 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 10:11:30.684511 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260811 10:11:30.685990 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:30.690992 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:30.687566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54911 (local address 127.0.0.1:45559)
0811 10:11:30.687614 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:30.687616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:30.687709 (+ 93us) server_negotiation.cc:407] Connection header received
0811 10:11:30.687767 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:30.687768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:30.687777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:30.687824 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:30.687970 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:30.688545 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:30.689668 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:30.689889 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:30.689950 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:30.690705 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:30.690706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:30.690707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:30.690712 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:30.690727 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:30.690732 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:30.690797 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:30.690871 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 10:11:30.690896 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:11:30.694314 13885 catalog_manager.cc:1347] Generated new cluster ID: f4dc8d4905634477a3dda1c6bdbebe67
I20260811 10:11:30.694379 13885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:30.711433 13885 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:30.712210 13885 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:30.721856 13885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c7d93d99ed24e149788545b4454d9c0: Generated new TSK 0
I20260811 10:11:30.722098 13885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:30.725809 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13792
2026-08-11T10:11:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:30 dist-test-slave-g350 krb5kdc[13910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:30 dist-test-slave-g350 krb5kdc[13910](info): set up 2 sockets
Aug 11 10:11:30 dist-test-slave-g350 krb5kdc[13910](info): commencing operation
krb5kdc: starting...
W20260811 10:11:31.844762 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:31 dist-test-slave-g350 krb5kdc[13910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:31.985431 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:32.070631 13926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:32.070856 13926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:32.070880 13926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:32.072093 13926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:32.072144 13926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:32.072158 13926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:32.072171 13926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:32.072183 13926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:32.073812 13926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:32.074077 13926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:32.074335 13926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:11:32.079788 13926 server_base.cc:1034] running on GCE node
W20260811 10:11:32.079738 13932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:32.079732 13934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:32.079762 13931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:32.080200 13926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:32.080513 13926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:32.081671 13926 hybrid_clock.cc:648] HybridClock initialized: now 1786443092081660 us; error 36 us; skew 500 ppm
Aug 11 10:11:32 dist-test-slave-g350 krb5kdc[13910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:32.084474 13926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:32.085005 13926 webserver.cc:460] Webserver started at http://127.0.0.1:37899/ using document root <none> and password file <none>
I20260811 10:11:32.085266 13926 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:32.085340 13926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:32.085485 13926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:32.086676 13926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "72d68b1e175d4a809a14f6b4c5a12077"
format_stamp: "Formatted at 2026-08-11 10:11:32 on dist-test-slave-g350"
server_key: "3d7981a288e4546c8213319c828024e4"
server_key_iv: "152d0bb9bfd49c8ae94cf3b561027080"
server_key_version: "encryptionkey@0"
I20260811 10:11:32.087054 13926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "72d68b1e175d4a809a14f6b4c5a12077"
format_stamp: "Formatted at 2026-08-11 10:11:32 on dist-test-slave-g350"
server_key: "3d7981a288e4546c8213319c828024e4"
server_key_iv: "152d0bb9bfd49c8ae94cf3b561027080"
server_key_version: "encryptionkey@0"
I20260811 10:11:32.088505 13926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:11:32.089407 13941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:32.089644 13926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:32.089759 13926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "72d68b1e175d4a809a14f6b4c5a12077"
format_stamp: "Formatted at 2026-08-11 10:11:32 on dist-test-slave-g350"
server_key: "3d7981a288e4546c8213319c828024e4"
server_key_iv: "152d0bb9bfd49c8ae94cf3b561027080"
server_key_version: "encryptionkey@0"
I20260811 10:11:32.089844 13926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:32.093357 13926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:32.095835 13926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:32.096004 13926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:32.100230 13926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37999
I20260811 10:11:32.100265 13998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37999 every 8 connection(s)
I20260811 10:11:32.100618 13926 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:32.101562 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 13926
I20260811 10:11:32.101706 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:32.101759 13999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:32.101917 23236 external_mini_cluster.cc:1428] Setting key 1753ab88a2ce7e46a8391bb6a8aa0ece
I20260811 10:11:32.104569 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:32.102461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40991 (local address 127.0.0.1:37999)
0811 10:11:32.102695 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:32.102702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:32.102728 (+ 26us) server_negotiation.cc:407] Connection header received
0811 10:11:32.103227 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:32.103234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:32.103258 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:32.103398 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:32.104097 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:32.104100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:32.104100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:32.104104 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:32.104141 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:32.104151 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:32.104205 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:32.104307 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 10:11:32.104361 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260811 10:11:32.105409 13999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077: Bootstrap starting.
I20260811 10:11:32.105536 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:32.106127 13999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:32.106420 13999 log.cc:824] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:32.107317 13999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077: No bootstrap required, opened a new log
I20260811 10:11:32.108119 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:32.107061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46051 (local address 127.0.0.1:37999)
0811 10:11:32.107101 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:32.107103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:32.107216 (+ 113us) server_negotiation.cc:407] Connection header received
0811 10:11:32.107246 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:32.107246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:32.107251 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:32.107324 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:32.107915 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:32.107917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:32.107917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:32.107921 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:32.107938 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:32.107942 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:32.108008 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:32.108041 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 10:11:32.108054 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:32.109578 13999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d68b1e175d4a809a14f6b4c5a12077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260811 10:11:32.109818 13999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:32.109848 13999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d68b1e175d4a809a14f6b4c5a12077, State: Initialized, Role: FOLLOWER
I20260811 10:11:32.109937 13999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d68b1e175d4a809a14f6b4c5a12077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260811 10:11:32.110000 13999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:32.110021 13999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:32.110057 13999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:32.110586 13999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d68b1e175d4a809a14f6b4c5a12077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260811 10:11:32.110677 13999 leader_election.cc:302] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72d68b1e175d4a809a14f6b4c5a12077; no voters:
I20260811 10:11:32.110814 13999 leader_election.cc:288] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:32.110888 14009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:32.111047 14009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [term 1 LEADER]: Becoming Leader. State: Replica: 72d68b1e175d4a809a14f6b4c5a12077, State: Running, Role: LEADER
I20260811 10:11:32.111048 13999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:32.111150 14009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d68b1e175d4a809a14f6b4c5a12077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260811 10:11:32.111554 14011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72d68b1e175d4a809a14f6b4c5a12077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d68b1e175d4a809a14f6b4c5a12077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } } }
I20260811 10:11:32.111562 14012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72d68b1e175d4a809a14f6b4c5a12077. Latest consensus state: current_term: 1 leader_uuid: "72d68b1e175d4a809a14f6b4c5a12077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d68b1e175d4a809a14f6b4c5a12077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } } }
I20260811 10:11:32.111734 14011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:32.111884 14012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:32.112241 14020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:32.112676 14020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:32.114629 14020 catalog_manager.cc:1347] Generated new cluster ID: 6922a3a3133e4786a2a9b3b77c56c296
I20260811 10:11:32.114688 14020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:32.124224 14020 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:32.124483 14020 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:32.128947 14020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72d68b1e175d4a809a14f6b4c5a12077: Generated new TSK 0
I20260811 10:11:32.129156 14020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:32.131999 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 13926
2026-08-11T10:11:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:32 dist-test-slave-g350 krb5kdc[14032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:32 dist-test-slave-g350 krb5kdc[14032](info): set up 2 sockets
Aug 11 10:11:32 dist-test-slave-g350 krb5kdc[14032](info): commencing operation
krb5kdc: starting...
W20260811 10:11:33.249039 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:33 dist-test-slave-g350 krb5kdc[14032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:33.388513 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:33.477481 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:33.477825 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:33.477869 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:33.479095 14047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:33.479274 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:33.479336 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:33.479365 14047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:33.479394 14047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:33.481143 14047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:33.481463 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:33.481768 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:11:33.487068 14047 server_base.cc:1034] running on GCE node
W20260811 10:11:33.487010 14056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:33.487069 14055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:33.487010 14059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:33.487701 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:33.488080 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:33.489287 14047 hybrid_clock.cc:648] HybridClock initialized: now 1786443093489264 us; error 49 us; skew 500 ppm
Aug 11 10:11:33 dist-test-slave-g350 krb5kdc[14032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443093, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:33.492280 14047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:33.492772 14047 webserver.cc:460] Webserver started at http://127.0.0.1:42253/ using document root <none> and password file <none>
I20260811 10:11:33.493011 14047 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:33.493058 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:33.493239 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:33.494423 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "6877a26786014300b8c6225e0c5c9c33"
format_stamp: "Formatted at 2026-08-11 10:11:33 on dist-test-slave-g350"
server_key: "398b549bd17ee8f1f730385b3fa456d6"
server_key_iv: "77333597f480442bdfa7c72e5f5f716b"
server_key_version: "encryptionkey@0"
I20260811 10:11:33.494791 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "6877a26786014300b8c6225e0c5c9c33"
format_stamp: "Formatted at 2026-08-11 10:11:33 on dist-test-slave-g350"
server_key: "398b549bd17ee8f1f730385b3fa456d6"
server_key_iv: "77333597f480442bdfa7c72e5f5f716b"
server_key_version: "encryptionkey@0"
I20260811 10:11:33.496537 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 10:11:33.497932 14069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:33.498191 14047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:11:33.498410 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "6877a26786014300b8c6225e0c5c9c33"
format_stamp: "Formatted at 2026-08-11 10:11:33 on dist-test-slave-g350"
server_key: "398b549bd17ee8f1f730385b3fa456d6"
server_key_iv: "77333597f480442bdfa7c72e5f5f716b"
server_key_version: "encryptionkey@0"
I20260811 10:11:33.498533 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:33.518816 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:33.521541 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:33.521741 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:33.526001 14047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45405
I20260811 10:11:33.526106 14126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45405 every 8 connection(s)
I20260811 10:11:33.526471 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:33.527508 14127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:33.530010 14127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33: Bootstrap starting.
I20260811 10:11:33.530690 14127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:33.531006 14127 log.cc:824] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:33.531841 14127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33: No bootstrap required, opened a new log
I20260811 10:11:33.534080 14127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6877a26786014300b8c6225e0c5c9c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45405 } }
I20260811 10:11:33.534368 14127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:33.534418 14127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6877a26786014300b8c6225e0c5c9c33, State: Initialized, Role: FOLLOWER
I20260811 10:11:33.534622 14127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6877a26786014300b8c6225e0c5c9c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45405 } }
I20260811 10:11:33.534740 14127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:33.534788 14127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:33.534853 14127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:33.534999 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14047
I20260811 10:11:33.535135 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:33.535317 23236 external_mini_cluster.cc:1428] Setting key 13a17eb1fb54c2dbdd1a1271158e7cfc
I20260811 10:11:33.535746 14127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6877a26786014300b8c6225e0c5c9c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45405 } }
I20260811 10:11:33.535897 14127 leader_election.cc:302] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6877a26786014300b8c6225e0c5c9c33; no voters:
I20260811 10:11:33.536317 14132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:33.536360 14127 leader_election.cc:288] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:33.536475 14132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [term 1 LEADER]: Becoming Leader. State: Replica: 6877a26786014300b8c6225e0c5c9c33, State: Running, Role: LEADER
I20260811 10:11:33.536695 14127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:33.536695 14132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6877a26786014300b8c6225e0c5c9c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45405 } }
I20260811 10:11:33.538637 14134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6877a26786014300b8c6225e0c5c9c33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6877a26786014300b8c6225e0c5c9c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45405 } } }
I20260811 10:11:33.538702 14134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:33.538478 14133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6877a26786014300b8c6225e0c5c9c33. Latest consensus state: current_term: 1 leader_uuid: "6877a26786014300b8c6225e0c5c9c33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6877a26786014300b8c6225e0c5c9c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45405 } } }
I20260811 10:11:33.539052 14133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33 [sys.catalog]: This master's current role is: LEADER
W20260811 10:11:33.539532 14147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:33.539588 14147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:33.539639 14150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:33.539808 14150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:33.542037 14150 catalog_manager.cc:1347] Generated new cluster ID: 677d87fbf2134613ab5f0472f86a9c55
I20260811 10:11:33.542097 14150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:11:33 dist-test-slave-g350 krb5kdc[14032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:11:33.548305 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:33.535922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:35315 (local address 127.0.0.1:45405)
0811 10:11:33.536092 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:33.536104 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:33.536191 (+ 87us) server_negotiation.cc:407] Connection header received
0811 10:11:33.536373 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:33.536381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:33.536455 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:33.537508 (+ 1053us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:33.537651 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:33.539319 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:33.540350 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:33.540979 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:33.541120 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:33.544036 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:33.544040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:33.544041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:33.544054 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:33.546481 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:33.546936 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:33.546945 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:33.546945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:33.547054 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:33.547320 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:33.547321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:33.547322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:33.547582 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:33.547770 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:33.547898 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 10:11:33.548117 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260811 10:11:33.549346 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:33.561417 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:33.551041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:41609 (local address 127.0.0.1:45405)
0811 10:11:33.551083 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:33.551085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:33.553924 (+ 2839us) server_negotiation.cc:407] Connection header received
0811 10:11:33.554039 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:33.554040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:33.554052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:33.554104 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:33.554796 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:33.555527 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:33.556254 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:33.556494 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:33.556567 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:33.558344 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:33.558345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:33.558345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:33.558354 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:33.559989 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:33.560430 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:33.560432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:33.560432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:33.560499 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:33.560758 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:33.560760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:33.560760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:33.560889 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:33.561033 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:33.561160 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 10:11:33.561268 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:11:33.565932 14150 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:33.566774 14150 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:33.569394 14150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6877a26786014300b8c6225e0c5c9c33: Generated new TSK 0
I20260811 10:11:33.569638 14150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:33.573846 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14047
W20260811 10:11:33.583307 14030 connection.cc:535] client connection to 127.0.0.1:45405 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45405): Connection reset by peer (error 104)
2026-08-11T10:11:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:33 dist-test-slave-g350 krb5kdc[14161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:33 dist-test-slave-g350 krb5kdc[14161](info): set up 2 sockets
Aug 11 10:11:33 dist-test-slave-g350 krb5kdc[14161](info): commencing operation
krb5kdc: starting...
W20260811 10:11:34.757386 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:34 dist-test-slave-g350 krb5kdc[14161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:34.894742 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:34.981165 14177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:34.981388 14177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:34.981412 14177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:34.982677 14177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:34.982722 14177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:34.982738 14177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:34.982750 14177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:34.982764 14177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:34.984325 14177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:34.984602 14177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:34.984872 14177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:34.990165 14185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:34.990173 14182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:34.990293 14183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:34.990204 14177 server_base.cc:1034] running on GCE node
I20260811 10:11:34.990720 14177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:34.991050 14177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:34.992221 14177 hybrid_clock.cc:648] HybridClock initialized: now 1786443094992207 us; error 32 us; skew 500 ppm
Aug 11 10:11:34 dist-test-slave-g350 krb5kdc[14161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443094, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:34.994980 14177 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:34.995465 14177 webserver.cc:460] Webserver started at http://127.0.0.1:46353/ using document root <none> and password file <none>
I20260811 10:11:34.995712 14177 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:34.995790 14177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:34.995949 14177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:34.997035 14177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "46f80d11b13b4de0961d452fa30dcc92"
format_stamp: "Formatted at 2026-08-11 10:11:34 on dist-test-slave-g350"
server_key: "d1ed755d6a182ee26c2096d2c0705aa7"
server_key_iv: "12dfb62c6f5b91242b4b55a59262e90c"
server_key_version: "encryptionkey@0"
I20260811 10:11:34.997411 14177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "46f80d11b13b4de0961d452fa30dcc92"
format_stamp: "Formatted at 2026-08-11 10:11:34 on dist-test-slave-g350"
server_key: "d1ed755d6a182ee26c2096d2c0705aa7"
server_key_iv: "12dfb62c6f5b91242b4b55a59262e90c"
server_key_version: "encryptionkey@0"
I20260811 10:11:34.998883 14177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:34.999696 14194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:34.999900 14177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:11:35.000007 14177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "46f80d11b13b4de0961d452fa30dcc92"
format_stamp: "Formatted at 2026-08-11 10:11:34 on dist-test-slave-g350"
server_key: "d1ed755d6a182ee26c2096d2c0705aa7"
server_key_iv: "12dfb62c6f5b91242b4b55a59262e90c"
server_key_version: "encryptionkey@0"
I20260811 10:11:35.000108 14177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:35.033087 14177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:35.035208 14177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:35.035537 14177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:35.039987 14177 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44897
I20260811 10:11:35.040025 14253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44897 every 8 connection(s)
I20260811 10:11:35.040401 14177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:35.041182 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14177
I20260811 10:11:35.041312 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:35.041524 23236 external_mini_cluster.cc:1428] Setting key fbc75f77403204c8460abcf8ea5a708d
I20260811 10:11:35.041661 14255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:35.044941 14255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92: Bootstrap starting.
I20260811 10:11:35.045653 14255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:35.045957 14255 log.cc:824] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:35.046701 14255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92: No bootstrap required, opened a new log
I20260811 10:11:35.047192 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:35.042145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:45595 (local address 127.0.0.1:44897)
0811 10:11:35.042357 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:35.042366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:35.042398 (+ 32us) server_negotiation.cc:407] Connection header received
0811 10:11:35.042815 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:35.042823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:35.042870 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:35.042982 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:35.043236 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:35.044162 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:35.045232 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:35.045718 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:35.045847 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:35.046542 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:35.046546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:35.046546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:35.046713 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:35.046741 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:35.046752 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:35.046896 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:35.046979 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:11:35.047014 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260811 10:11:35.048192 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:35.048983 14255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f80d11b13b4de0961d452fa30dcc92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260811 10:11:35.049188 14255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:35.049239 14255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f80d11b13b4de0961d452fa30dcc92, State: Initialized, Role: FOLLOWER
I20260811 10:11:35.049368 14255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f80d11b13b4de0961d452fa30dcc92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260811 10:11:35.049463 14255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:35.049500 14255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:35.049553 14255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:35.050263 14255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f80d11b13b4de0961d452fa30dcc92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260811 10:11:35.050355 14255 leader_election.cc:302] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46f80d11b13b4de0961d452fa30dcc92; no voters:
I20260811 10:11:35.050534 14255 leader_election.cc:288] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:35.050597 14267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:35.050761 14267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [term 1 LEADER]: Becoming Leader. State: Replica: 46f80d11b13b4de0961d452fa30dcc92, State: Running, Role: LEADER
I20260811 10:11:35.050807 14255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:35.050870 14267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f80d11b13b4de0961d452fa30dcc92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260811 10:11:35.051182 14267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f80d11b13b4de0961d452fa30dcc92. Latest consensus state: current_term: 1 leader_uuid: "46f80d11b13b4de0961d452fa30dcc92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f80d11b13b4de0961d452fa30dcc92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } } }
I20260811 10:11:35.051192 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46f80d11b13b4de0961d452fa30dcc92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f80d11b13b4de0961d452fa30dcc92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } } }
I20260811 10:11:35.051275 14267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:35.051326 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:35.052089 14276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:35.052654 14276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:35.053411 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:35.049519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:48001 (local address 127.0.0.1:44897)
0811 10:11:35.049557 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:35.049559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:35.049725 (+ 166us) server_negotiation.cc:407] Connection header received
0811 10:11:35.049792 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:35.049793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:35.049801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:35.049848 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:35.049977 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:35.050507 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:35.051920 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:35.052192 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:35.052238 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:35.053129 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:35.053131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:35.053131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:35.053136 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:35.053152 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:35.053157 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:35.053227 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:35.053323 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 10:11:35.053334 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:11:35.054833 14276 catalog_manager.cc:1347] Generated new cluster ID: e45b93d8baf14610a2de6e2679f1d67f
I20260811 10:11:35.054888 14276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:35.068647 14276 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:35.069270 14276 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:35.073448 14276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46f80d11b13b4de0961d452fa30dcc92: Generated new TSK 0
I20260811 10:11:35.073660 14276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:35.076347 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14177
W20260811 10:11:35.084715 14159 connection.cc:535] client connection to 127.0.0.1:44897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44897): Connection reset by peer (error 104)
2026-08-11T10:11:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:35 dist-test-slave-g350 krb5kdc[14292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:35 dist-test-slave-g350 krb5kdc[14292](info): set up 2 sockets
Aug 11 10:11:35 dist-test-slave-g350 krb5kdc[14292](info): commencing operation
krb5kdc: starting...
W20260811 10:11:36.238408 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:36 dist-test-slave-g350 krb5kdc[14292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:36.377285 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:36.462837 14306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:36.463100 14306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:36.463148 14306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:36.464418 14306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:36.464493 14306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:36.464519 14306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:36.464545 14306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:36.464602 14306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:36.466164 14306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:36.466533 14306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:36.466791 14306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:36.472040 14317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:36.472098 14315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:36.472182 14306 server_base.cc:1034] running on GCE node
W20260811 10:11:36.472044 14314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:36.472601 14306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:36.472944 14306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:36.474117 14306 hybrid_clock.cc:648] HybridClock initialized: now 1786443096474103 us; error 49 us; skew 500 ppm
Aug 11 10:11:36 dist-test-slave-g350 krb5kdc[14292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:36.481912 14306 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:36.482412 14306 webserver.cc:460] Webserver started at http://127.0.0.1:36393/ using document root <none> and password file <none>
I20260811 10:11:36.482659 14306 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:36.482733 14306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:36.482903 14306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:36.483978 14306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9"
format_stamp: "Formatted at 2026-08-11 10:11:36 on dist-test-slave-g350"
server_key: "2d60481350da80913ed681fb485cca8c"
server_key_iv: "9fc62b533d6054dfbb640e94b74f1835"
server_key_version: "encryptionkey@0"
I20260811 10:11:36.484283 14306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9"
format_stamp: "Formatted at 2026-08-11 10:11:36 on dist-test-slave-g350"
server_key: "2d60481350da80913ed681fb485cca8c"
server_key_iv: "9fc62b533d6054dfbb640e94b74f1835"
server_key_version: "encryptionkey@0"
I20260811 10:11:36.485702 14306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:36.486503 14324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:36.486694 14306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:36.486810 14306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9"
format_stamp: "Formatted at 2026-08-11 10:11:36 on dist-test-slave-g350"
server_key: "2d60481350da80913ed681fb485cca8c"
server_key_iv: "9fc62b533d6054dfbb640e94b74f1835"
server_key_version: "encryptionkey@0"
I20260811 10:11:36.486889 14306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:36.490345 14306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:36.492520 14306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:36.492683 14306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:36.496920 14306 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39863
I20260811 10:11:36.496982 14384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39863 every 8 connection(s)
I20260811 10:11:36.497321 14306 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:36.498503 14385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:36.501031 14385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9: Bootstrap starting.
I20260811 10:11:36.501752 14385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:36.502071 14385 log.cc:824] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:36.502864 14385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9: No bootstrap required, opened a new log
I20260811 10:11:36.503197 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14306
I20260811 10:11:36.503337 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:36.503525 23236 external_mini_cluster.cc:1428] Setting key 074a62397af0aabb14fcabd16276e0a6
I20260811 10:11:36.505286 14385 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260811 10:11:36.505467 14385 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:36.505529 14385 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5db3e146fc6479e8dbf01c2b8fb2dd9, State: Initialized, Role: FOLLOWER
I20260811 10:11:36.505714 14385 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260811 10:11:36.505825 14385 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:36.505865 14385 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:36.505916 14385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:36.506319 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:36.504112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:51539 (local address 127.0.0.1:39863)
0811 10:11:36.504322 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:36.504332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:36.504365 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:11:36.504828 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:36.504836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:36.504869 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:36.505028 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:36.505704 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:36.505707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:36.505708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:36.505712 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:36.505758 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:36.505773 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:36.505852 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:36.506023 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:11:36.506077 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260811 10:11:36.506927 14385 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260811 10:11:36.507045 14385 leader_election.cc:302] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5db3e146fc6479e8dbf01c2b8fb2dd9; no voters:
I20260811 10:11:36.507218 14385 leader_election.cc:288] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:36.507241 14390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:36.507356 14390 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [term 1 LEADER]: Becoming Leader. State: Replica: e5db3e146fc6479e8dbf01c2b8fb2dd9, State: Running, Role: LEADER
I20260811 10:11:36.507434 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:36.507436 14390 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260811 10:11:36.507477 14385 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:36.507757 14392 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5db3e146fc6479e8dbf01c2b8fb2dd9. Latest consensus state: current_term: 1 leader_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } } }
I20260811 10:11:36.507848 14392 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:36.508030 14391 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db3e146fc6479e8dbf01c2b8fb2dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } } }
I20260811 10:11:36.508075 14391 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:36.508086 14395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:36.508610 14395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:36.510962 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:36.509217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:60339 (local address 127.0.0.1:39863)
0811 10:11:36.509257 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:36.509259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:36.510063 (+ 804us) server_negotiation.cc:407] Connection header received
0811 10:11:36.510091 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:36.510093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:36.510098 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:36.510164 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:36.510766 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:36.510767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:36.510767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:36.510771 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:36.510787 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:36.510791 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:36.510853 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:36.510893 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 10:11:36.510906 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:11:36.514153 14395 catalog_manager.cc:1347] Generated new cluster ID: 83d47c4491ab49de8e88d31f09a08fcd
I20260811 10:11:36.514202 14395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:36.523494 14395 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:36.523679 14395 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:36.532449 14395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5db3e146fc6479e8dbf01c2b8fb2dd9: Generated new TSK 0
I20260811 10:11:36.532696 14395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:36.536638 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14306
2026-08-11T10:11:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:36 dist-test-slave-g350 krb5kdc[14426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:36 dist-test-slave-g350 krb5kdc[14426](info): set up 2 sockets
Aug 11 10:11:36 dist-test-slave-g350 krb5kdc[14426](info): commencing operation
krb5kdc: starting...
W20260811 10:11:37.653975 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:37 dist-test-slave-g350 krb5kdc[14426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:37.794353 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41337
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:37.881621 14439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:37.881860 14439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:37.881884 14439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:37.883075 14439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:37.883121 14439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:37.883136 14439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:37.883149 14439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:37.883163 14439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:37.884681 14439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41337
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:37.884898 14439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:37.885085 14439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:37.890497 14444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:37.890515 14445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:37.890506 14447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:37.890725 14439 server_base.cc:1034] running on GCE node
I20260811 10:11:37.891036 14439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:37.891346 14439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:37.892549 14439 hybrid_clock.cc:648] HybridClock initialized: now 1786443097892535 us; error 55 us; skew 500 ppm
Aug 11 10:11:37 dist-test-slave-g350 krb5kdc[14426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:37.895460 14439 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:37.896000 14439 webserver.cc:460] Webserver started at http://127.0.0.1:38375/ using document root <none> and password file <none>
I20260811 10:11:37.896262 14439 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:37.896333 14439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:37.896490 14439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:37.897715 14439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "ee31c79ee44541e9aa118b86f8c8cb36"
format_stamp: "Formatted at 2026-08-11 10:11:37 on dist-test-slave-g350"
server_key: "69be4642120e22a10951d1e10e22d7a8"
server_key_iv: "5087a05fad39837ab559f44dd8d1679b"
server_key_version: "encryptionkey@0"
I20260811 10:11:37.898108 14439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "ee31c79ee44541e9aa118b86f8c8cb36"
format_stamp: "Formatted at 2026-08-11 10:11:37 on dist-test-slave-g350"
server_key: "69be4642120e22a10951d1e10e22d7a8"
server_key_iv: "5087a05fad39837ab559f44dd8d1679b"
server_key_version: "encryptionkey@0"
I20260811 10:11:37.899648 14439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:11:37.900548 14454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:37.900763 14439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:37.900903 14439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ee31c79ee44541e9aa118b86f8c8cb36"
format_stamp: "Formatted at 2026-08-11 10:11:37 on dist-test-slave-g350"
server_key: "69be4642120e22a10951d1e10e22d7a8"
server_key_iv: "5087a05fad39837ab559f44dd8d1679b"
server_key_version: "encryptionkey@0"
I20260811 10:11:37.900985 14439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:37.909909 14439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:37.912297 14439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:37.912472 14439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:37.916981 14439 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41337
I20260811 10:11:37.917025 14512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41337 every 8 connection(s)
I20260811 10:11:37.917423 14439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:37.918478 14513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:37.920888 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14439
I20260811 10:11:37.920970 14513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36: Bootstrap starting.
I20260811 10:11:37.920991 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:37.921204 23236 external_mini_cluster.cc:1428] Setting key 43946c683824088b237bfbcb2408fd82
I20260811 10:11:37.922446 14513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:37.922770 14513 log.cc:824] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:37.923745 14513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36: No bootstrap required, opened a new log
I20260811 10:11:37.926337 14513 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41337 } }
I20260811 10:11:37.926527 14513 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:37.926573 14513 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee31c79ee44541e9aa118b86f8c8cb36, State: Initialized, Role: FOLLOWER
I20260811 10:11:37.926767 14513 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41337 } }
I20260811 10:11:37.926877 14513 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:37.926931 14513 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:37.926992 14513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:37.927863 14513 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41337 } }
I20260811 10:11:37.928000 14513 leader_election.cc:302] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee31c79ee44541e9aa118b86f8c8cb36; no voters:
I20260811 10:11:37.928208 14513 leader_election.cc:288] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:37.928236 14518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:37.928449 14518 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [term 1 LEADER]: Becoming Leader. State: Replica: ee31c79ee44541e9aa118b86f8c8cb36, State: Running, Role: LEADER
I20260811 10:11:37.928548 14513 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:37.928586 14518 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41337 } }
I20260811 10:11:37.929067 14519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41337 } } }
I20260811 10:11:37.929144 14519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:37.929354 14520 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee31c79ee44541e9aa118b86f8c8cb36. Latest consensus state: current_term: 1 leader_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee31c79ee44541e9aa118b86f8c8cb36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41337 } } }
I20260811 10:11:37.929692 14520 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36 [sys.catalog]: This master's current role is: LEADER
Aug 11 10:11:37 dist-test-slave-g350 krb5kdc[14426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260811 10:11:37.930639 14535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:37.930743 14535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:37.930824 14525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:37.930987 14525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:37.932893 14525 catalog_manager.cc:1347] Generated new cluster ID: 4f8ab84e388a48eeb99bbed933be1faa
I20260811 10:11:37.932950 14525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:37.934993 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:37.921805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:49539 (local address 127.0.0.1:41337)
0811 10:11:37.922425 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:37.922433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:37.922522 (+ 89us) server_negotiation.cc:407] Connection header received
0811 10:11:37.923150 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:37.923159 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:37.923227 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:37.923370 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:37.924314 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:37.925356 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:37.926643 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:37.927246 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:37.927339 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:37.930756 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:37.930764 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:37.930765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:37.930778 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:37.933174 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:37.933682 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:37.933686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:37.933686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:37.933786 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:37.934065 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:37.934066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:37.934067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:37.934329 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:37.934533 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:37.934648 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 10:11:37.934770 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":487,"threads_started":1}
I20260811 10:11:37.938514 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:37.952157 14525 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:37.953116 14525 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:37.959223 14525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee31c79ee44541e9aa118b86f8c8cb36: Generated new TSK 0
I20260811 10:11:37.959568 14525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:37.960930 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:37.943867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:45883 (local address 127.0.0.1:41337)
0811 10:11:37.943943 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:37.943945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:37.951907 (+ 7962us) server_negotiation.cc:407] Connection header received
0811 10:11:37.952023 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:37.952024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:37.952036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:37.952067 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:37.952897 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:37.953702 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:37.954842 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:37.955211 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:37.955255 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:37.957322 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:37.957324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:37.957325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:37.957334 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:37.959261 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:37.959805 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:37.959806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:37.959807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:37.959879 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:37.960164 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:37.960166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:37.960166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:37.960351 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:37.960488 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:37.960710 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 10:11:37.960803 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:11:37.963733 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14439
W20260811 10:11:37.977439 14424 connection.cc:535] client connection to 127.0.0.1:41337 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41337): Connection reset by peer (error 104)
2026-08-11T10:11:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:38 dist-test-slave-g350 krb5kdc[14549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:38 dist-test-slave-g350 krb5kdc[14549](info): set up 2 sockets
Aug 11 10:11:38 dist-test-slave-g350 krb5kdc[14549](info): commencing operation
krb5kdc: starting...
W20260811 10:11:39.152058 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:39 dist-test-slave-g350 krb5kdc[14549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:39.289655 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:39.375522 14562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:39.375787 14562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:39.375831 14562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:39.377170 14562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:39.377250 14562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:39.377275 14562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:39.377298 14562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:39.377326 14562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:39.378913 14562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42629
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:39.379220 14562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:39.379462 14562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:39.384714 14567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:39.384714 14571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:39.384708 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:39.385013 14562 server_base.cc:1034] running on GCE node
I20260811 10:11:39.385280 14562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:39.385620 14562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:39.386797 14562 hybrid_clock.cc:648] HybridClock initialized: now 1786443099386767 us; error 45 us; skew 500 ppm
Aug 11 10:11:39 dist-test-slave-g350 krb5kdc[14549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:39.389519 14562 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:39.390012 14562 webserver.cc:460] Webserver started at http://127.0.0.1:44701/ using document root <none> and password file <none>
I20260811 10:11:39.390205 14562 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:39.390246 14562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:39.390368 14562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:39.391474 14562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "7f772f3ac35c47ab940293bc816a7a26"
format_stamp: "Formatted at 2026-08-11 10:11:39 on dist-test-slave-g350"
server_key: "e4c898679233c82bb6072c71edef6b90"
server_key_iv: "45501223fa5f2914847737ab9f9d9b21"
server_key_version: "encryptionkey@0"
I20260811 10:11:39.391790 14562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "7f772f3ac35c47ab940293bc816a7a26"
format_stamp: "Formatted at 2026-08-11 10:11:39 on dist-test-slave-g350"
server_key: "e4c898679233c82bb6072c71edef6b90"
server_key_iv: "45501223fa5f2914847737ab9f9d9b21"
server_key_version: "encryptionkey@0"
I20260811 10:11:39.393244 14562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:39.394147 14579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:39.394348 14562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:11:39.394454 14562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "7f772f3ac35c47ab940293bc816a7a26"
format_stamp: "Formatted at 2026-08-11 10:11:39 on dist-test-slave-g350"
server_key: "e4c898679233c82bb6072c71edef6b90"
server_key_iv: "45501223fa5f2914847737ab9f9d9b21"
server_key_version: "encryptionkey@0"
I20260811 10:11:39.394515 14562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:39.404114 14562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:39.406865 14562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:39.406987 14562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:39.411059 14562 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42629
I20260811 10:11:39.411114 14637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42629 every 8 connection(s)
I20260811 10:11:39.411450 14562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:39.412500 14638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:39.414978 14638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26: Bootstrap starting.
I20260811 10:11:39.415647 14638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:39.415843 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14562
I20260811 10:11:39.415997 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:39.416036 14638 log.cc:824] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:39.416234 23236 external_mini_cluster.cc:1428] Setting key cee2b24db819e2019c2d065bc7c541ba
I20260811 10:11:39.416946 14638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26: No bootstrap required, opened a new log
I20260811 10:11:39.419844 14638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f772f3ac35c47ab940293bc816a7a26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260811 10:11:39.420024 14638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:39.420056 14638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f772f3ac35c47ab940293bc816a7a26, State: Initialized, Role: FOLLOWER
I20260811 10:11:39.420172 14638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f772f3ac35c47ab940293bc816a7a26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260811 10:11:39.420252 14638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:39.420291 14638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:39.420363 14638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:39.421000 14638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f772f3ac35c47ab940293bc816a7a26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260811 10:11:39.421128 14638 leader_election.cc:302] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f772f3ac35c47ab940293bc816a7a26; no voters:
I20260811 10:11:39.421340 14638 leader_election.cc:288] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:39.421442 14643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:39.421626 14638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:39.421694 14643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [term 1 LEADER]: Becoming Leader. State: Replica: 7f772f3ac35c47ab940293bc816a7a26, State: Running, Role: LEADER
I20260811 10:11:39.421815 14643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f772f3ac35c47ab940293bc816a7a26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260811 10:11:39.422168 14645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f772f3ac35c47ab940293bc816a7a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f772f3ac35c47ab940293bc816a7a26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } } }
I20260811 10:11:39.422156 14643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f772f3ac35c47ab940293bc816a7a26. Latest consensus state: current_term: 1 leader_uuid: "7f772f3ac35c47ab940293bc816a7a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f772f3ac35c47ab940293bc816a7a26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } } }
I20260811 10:11:39.422402 14643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:39.422385 14645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:39.422797 14655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:39.423558 14655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:39.423944 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:39.416866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:40891 (local address 127.0.0.1:42629)
0811 10:11:39.417052 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:39.417063 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:39.417104 (+ 41us) server_negotiation.cc:407] Connection header received
0811 10:11:39.418298 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:39.418307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:39.418376 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:39.418538 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:39.418821 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:39.420234 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:39.421231 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:39.421881 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:39.422888 (+ 1007us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:39.423052 (+ 164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:39.423056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:39.423056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:39.423287 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:39.423332 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:39.423347 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:39.423609 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:39.423660 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 10:11:39.423691 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260811 10:11:39.424943 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:39.425189 14655 catalog_manager.cc:1347] Generated new cluster ID: 66fca0f981b54c26a2dc1cc53974580d
I20260811 10:11:39.425237 14655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:39.430141 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:39.426576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:60245 (local address 127.0.0.1:42629)
0811 10:11:39.426618 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:39.426620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:39.426629 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:11:39.426653 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:39.426653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:39.426663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:39.426749 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:39.426906 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:39.427635 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:39.428619 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:39.428953 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:39.429028 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:39.429863 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:39.429864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:39.429865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:39.429869 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:39.429885 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:39.429889 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:39.429948 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:39.430051 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 10:11:39.430062 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":20}
I20260811 10:11:39.435477 14655 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:39.436045 14655 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:39.441946 14655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f772f3ac35c47ab940293bc816a7a26: Generated new TSK 0
I20260811 10:11:39.442152 14655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:39.449071 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14562
W20260811 10:11:39.456279 14546 connection.cc:535] client connection to 127.0.0.1:42629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42629): Connection reset by peer (error 104)
2026-08-11T10:11:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:39 dist-test-slave-g350 krb5kdc[14672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:39 dist-test-slave-g350 krb5kdc[14672](info): set up 2 sockets
Aug 11 10:11:39 dist-test-slave-g350 krb5kdc[14672](info): commencing operation
krb5kdc: starting...
W20260811 10:11:40.557260 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:40 dist-test-slave-g350 krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:11:40.700615 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:40.790045 14685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:40.790324 14685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:40.790364 14685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:40.791559 14685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:40.791637 14685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:40.791661 14685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:40.791682 14685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:40.791710 14685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:40.793319 14685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39213
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:40.793665 14685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:40.793917 14685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:40.799192 14694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:40.799302 14690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:40.799365 14685 server_base.cc:1034] running on GCE node
W20260811 10:11:40.799413 14692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:40.799695 14685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:40.799963 14685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:40.801123 14685 hybrid_clock.cc:648] HybridClock initialized: now 1786443100801101 us; error 32 us; skew 500 ppm
Aug 11 10:11:40 dist-test-slave-g350 krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:40.804026 14685 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:11:40.804545 14685 webserver.cc:460] Webserver started at http://127.0.0.1:40733/ using document root <none> and password file <none>
I20260811 10:11:40.804798 14685 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:40.804847 14685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:40.804993 14685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:40.806213 14685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "e1ed07538d194d24a20412dc1ce7837c"
format_stamp: "Formatted at 2026-08-11 10:11:40 on dist-test-slave-g350"
server_key: "3eac75682414010c764d43b509e44ef1"
server_key_iv: "c98bc922129cac936fc06e58cc935d76"
server_key_version: "encryptionkey@0"
I20260811 10:11:40.806654 14685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "e1ed07538d194d24a20412dc1ce7837c"
format_stamp: "Formatted at 2026-08-11 10:11:40 on dist-test-slave-g350"
server_key: "3eac75682414010c764d43b509e44ef1"
server_key_iv: "c98bc922129cac936fc06e58cc935d76"
server_key_version: "encryptionkey@0"
I20260811 10:11:40.808218 14685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:11:40.809252 14702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:40.809696 14685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:40.809819 14685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "e1ed07538d194d24a20412dc1ce7837c"
format_stamp: "Formatted at 2026-08-11 10:11:40 on dist-test-slave-g350"
server_key: "3eac75682414010c764d43b509e44ef1"
server_key_iv: "c98bc922129cac936fc06e58cc935d76"
server_key_version: "encryptionkey@0"
I20260811 10:11:40.809911 14685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:40.813508 14685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:40.816000 14685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:40.816165 14685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:40.820957 14685 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39213
I20260811 10:11:40.821015 14758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39213 every 8 connection(s)
I20260811 10:11:40.821414 14685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:40.822616 14759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:40.825280 14759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c: Bootstrap starting.
I20260811 10:11:40.826108 14759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:40.826454 14759 log.cc:824] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:40.827173 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14685
I20260811 10:11:40.827322 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:40.827541 23236 external_mini_cluster.cc:1428] Setting key 14865f420e3e2b265c67699f23ce64db
I20260811 10:11:40.827584 14759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c: No bootstrap required, opened a new log
I20260811 10:11:40.829811 14759 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ed07538d194d24a20412dc1ce7837c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260811 10:11:40.829990 14759 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:40.830015 14759 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ed07538d194d24a20412dc1ce7837c, State: Initialized, Role: FOLLOWER
I20260811 10:11:40.830183 14759 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ed07538d194d24a20412dc1ce7837c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260811 10:11:40.830296 14759 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:40.830343 14759 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:40.830399 14759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:40.831594 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:40.828236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:52427 (local address 127.0.0.1:39213)
0811 10:11:40.828480 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:40.828493 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:40.828538 (+ 45us) server_negotiation.cc:407] Connection header received
0811 10:11:40.829844 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:40.829854 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:40.829896 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:40.830132 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:40.831054 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:40.831058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:40.831059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:40.831064 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:40.831116 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:40.831129 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:40.831218 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:40.831358 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 10:11:40.831422 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":104,"threads_started":1}
I20260811 10:11:40.831653 14759 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ed07538d194d24a20412dc1ce7837c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260811 10:11:40.831820 14759 leader_election.cc:302] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1ed07538d194d24a20412dc1ce7837c; no voters:
I20260811 10:11:40.832018 14759 leader_election.cc:288] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:40.832113 14764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:40.832324 14764 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [term 1 LEADER]: Becoming Leader. State: Replica: e1ed07538d194d24a20412dc1ce7837c, State: Running, Role: LEADER
I20260811 10:11:40.832391 14759 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:40.832436 14764 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ed07538d194d24a20412dc1ce7837c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260811 10:11:40.832875 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:40.834022 14765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1ed07538d194d24a20412dc1ce7837c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ed07538d194d24a20412dc1ce7837c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } } }
I20260811 10:11:40.834149 14766 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1ed07538d194d24a20412dc1ce7837c. Latest consensus state: current_term: 1 leader_uuid: "e1ed07538d194d24a20412dc1ce7837c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ed07538d194d24a20412dc1ce7837c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } } }
I20260811 10:11:40.834270 14766 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:40.834167 14765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:40.834586 14791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:40.835232 14791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:40.836285 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:40.834731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:40207 (local address 127.0.0.1:39213)
0811 10:11:40.834772 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:40.834775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:40.835050 (+ 275us) server_negotiation.cc:407] Connection header received
0811 10:11:40.835132 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:40.835132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:40.835137 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:40.835220 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:40.835994 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:40.835996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:40.835997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:40.836002 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:40.836019 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:40.836024 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:40.836095 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:40.836172 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 10:11:40.836191 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:11:40.837112 14791 catalog_manager.cc:1347] Generated new cluster ID: fc7acbc560f9426a852e8e47545c607d
I20260811 10:11:40.837186 14791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:40.846751 14791 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:40.846997 14791 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:40.853852 14791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1ed07538d194d24a20412dc1ce7837c: Generated new TSK 0
I20260811 10:11:40.854112 14791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:40.860471 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14685
2026-08-11T10:11:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:40 dist-test-slave-g350 krb5kdc[14800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:40 dist-test-slave-g350 krb5kdc[14800](info): set up 2 sockets
Aug 11 10:11:40 dist-test-slave-g350 krb5kdc[14800](info): commencing operation
krb5kdc: starting...
W20260811 10:11:41.983593 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:42 dist-test-slave-g350 krb5kdc[14800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:42.122462 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37427
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:34571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:42.207549 14816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:42.207805 14816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:42.207849 14816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:42.209035 14816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:42.209112 14816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:42.209136 14816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:42.209168 14816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:42.209197 14816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:42.210808 14816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:34571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:42.211109 14816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:42.211351 14816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:42.217078 14822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:42.217084 14823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:42.217159 14826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:42.217432 14816 server_base.cc:1034] running on GCE node
I20260811 10:11:42.217739 14816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:42.218057 14816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:42.219245 14816 hybrid_clock.cc:648] HybridClock initialized: now 1786443102219212 us; error 47 us; skew 500 ppm
Aug 11 10:11:42 dist-test-slave-g350 krb5kdc[14800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443102, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:42.228593 14816 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:42.229130 14816 webserver.cc:460] Webserver started at http://127.22.177.62:36093/ using document root <none> and password file <none>
I20260811 10:11:42.229404 14816 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:42.229477 14816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:42.229681 14816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:42.230829 14816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f"
format_stamp: "Formatted at 2026-08-11 10:11:42 on dist-test-slave-g350"
server_key: "f1e7b93d7ce1a0270d85e8bc0a88b3a1"
server_key_iv: "dce301952aa22e40ad99eaa86f5ce923"
server_key_version: "encryptionkey@0"
I20260811 10:11:42.231197 14816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f"
format_stamp: "Formatted at 2026-08-11 10:11:42 on dist-test-slave-g350"
server_key: "f1e7b93d7ce1a0270d85e8bc0a88b3a1"
server_key_iv: "dce301952aa22e40ad99eaa86f5ce923"
server_key_version: "encryptionkey@0"
I20260811 10:11:42.232697 14816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:42.233656 14833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:42.233858 14816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:42.233978 14816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f"
format_stamp: "Formatted at 2026-08-11 10:11:42 on dist-test-slave-g350"
server_key: "f1e7b93d7ce1a0270d85e8bc0a88b3a1"
server_key_iv: "dce301952aa22e40ad99eaa86f5ce923"
server_key_version: "encryptionkey@0"
I20260811 10:11:42.234067 14816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:42.256405 14816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:42.259102 14816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:42.259310 14816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:42.264089 14816 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37427
I20260811 10:11:42.264149 14891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37427 every 8 connection(s)
I20260811 10:11:42.264556 14816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:42.265507 14892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:42.268056 14892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f: Bootstrap starting.
I20260811 10:11:42.268615 14892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:42.268863 14892 log.cc:824] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:42.269210 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14816
I20260811 10:11:42.269322 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:42.269567 23236 external_mini_cluster.cc:1428] Setting key dbcd931756cb8a0d27afc29620a2998b
I20260811 10:11:42.269913 14892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f: No bootstrap required, opened a new log
I20260811 10:11:42.271763 14892 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37427 } }
I20260811 10:11:42.271906 14892 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:42.271929 14892 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea36a8e5f7f64cabbfe0d4d4efbaad5f, State: Initialized, Role: FOLLOWER
I20260811 10:11:42.272089 14892 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37427 } }
I20260811 10:11:42.272184 14892 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:42.272219 14892 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:42.272266 14892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:42.273422 14892 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37427 } }
I20260811 10:11:42.273522 14892 leader_election.cc:302] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea36a8e5f7f64cabbfe0d4d4efbaad5f; no voters:
I20260811 10:11:42.273720 14892 leader_election.cc:288] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:42.273742 14897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:42.273885 14897 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [term 1 LEADER]: Becoming Leader. State: Replica: ea36a8e5f7f64cabbfe0d4d4efbaad5f, State: Running, Role: LEADER
I20260811 10:11:42.273975 14897 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37427 } }
I20260811 10:11:42.274039 14892 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:42.274327 14898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37427 } } }
I20260811 10:11:42.274582 14898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:42.274502 14899 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea36a8e5f7f64cabbfe0d4d4efbaad5f. Latest consensus state: current_term: 1 leader_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea36a8e5f7f64cabbfe0d4d4efbaad5f" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37427 } } }
I20260811 10:11:42.274874 14899 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:42.275041 14906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:42.275599 14906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:42.277611 14906 catalog_manager.cc:1347] Generated new cluster ID: ad58dd5c92074b26b8dd2c2974a38a52
I20260811 10:11:42.277674 14906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:11:42 dist-test-slave-g350 krb5kdc[14800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:42.283718 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:42.270262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35227 (local address 127.22.177.62:37427)
0811 10:11:42.270456 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:42.270465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:42.270498 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:11:42.271755 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:42.271764 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:42.271821 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:42.272010 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:42.272896 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:42.274290 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:42.275464 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:42.276113 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:42.276231 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:42.279422 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:42.279427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:42.279427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:42.279441 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:42.281698 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:42.282265 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:42.282269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:42.282269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:42.282419 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:42.282682 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:42.282683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:42.282683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:42.283014 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:42.283189 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:42.283379 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 10:11:42.283497 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260811 10:11:42.284963 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:42.290057 14906 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:42.290592 14906 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:42.300855 14906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea36a8e5f7f64cabbfe0d4d4efbaad5f: Generated new TSK 0
I20260811 10:11:42.301180 14906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:42.306247 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:42.286439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49021 (local address 127.22.177.62:37427)
0811 10:11:42.286473 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:42.286475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:42.289643 (+ 3168us) server_negotiation.cc:407] Connection header received
0811 10:11:42.289712 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:42.289713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:42.289725 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:42.289777 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:42.299080 (+ 9303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:42.299766 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:42.301146 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:42.301485 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:42.301537 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:42.303293 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:42.303295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:42.303296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:42.303304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:42.304890 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:42.305333 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:42.305334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:42.305335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:42.305387 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:42.305680 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:42.305681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:42.305681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:42.305808 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:42.305888 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:42.306071 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 10:11:42.306128 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 10:11:42.308627 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14816
W20260811 10:11:42.314649 14798 connection.cc:535] client connection to 127.22.177.62:37427 recv error: Network error: failed to read from TLS socket (remote: 127.22.177.62:37427): Connection reset by peer (error 104)
2026-08-11T10:11:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:42 dist-test-slave-g350 krb5kdc[14926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:42 dist-test-slave-g350 krb5kdc[14926](info): set up 2 sockets
Aug 11 10:11:42 dist-test-slave-g350 krb5kdc[14926](info): commencing operation
krb5kdc: starting...
W20260811 10:11:43.436964 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:43 dist-test-slave-g350 krb5kdc[14926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:43.578595 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40623
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:40623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:43.669305 14940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:43.669545 14940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:43.669571 14940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:43.670972 14940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:43.671029 14940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:43.671046 14940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:43.671058 14940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:43.671072 14940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:43.672765 14940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:40623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:40623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:43.673038 14940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:43.673293 14940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:43.678875 14948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:43.678843 14950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:43.678997 14940 server_base.cc:1034] running on GCE node
W20260811 10:11:43.678836 14947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:43.679324 14940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:43.679593 14940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:43.680766 14940 hybrid_clock.cc:648] HybridClock initialized: now 1786443103680737 us; error 43 us; skew 500 ppm
Aug 11 10:11:43 dist-test-slave-g350 krb5kdc[14926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443103, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:43.683617 14940 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:43.684033 14940 webserver.cc:460] Webserver started at http://127.22.177.62:45961/ using document root <none> and password file <none>
I20260811 10:11:43.684336 14940 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:43.684412 14940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:43.684576 14940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:43.685932 14940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "13ba079bfe9d4436aae43ce97d0a1456"
format_stamp: "Formatted at 2026-08-11 10:11:43 on dist-test-slave-g350"
server_key: "74ea7566ca02c3cea7765410c7c3757a"
server_key_iv: "dacf171941e5ae41f93b12f1635c2250"
server_key_version: "encryptionkey@0"
I20260811 10:11:43.686419 14940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "13ba079bfe9d4436aae43ce97d0a1456"
format_stamp: "Formatted at 2026-08-11 10:11:43 on dist-test-slave-g350"
server_key: "74ea7566ca02c3cea7765410c7c3757a"
server_key_iv: "dacf171941e5ae41f93b12f1635c2250"
server_key_version: "encryptionkey@0"
I20260811 10:11:43.687995 14940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:43.688905 14958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:43.689137 14940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:43.689254 14940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "13ba079bfe9d4436aae43ce97d0a1456"
format_stamp: "Formatted at 2026-08-11 10:11:43 on dist-test-slave-g350"
server_key: "74ea7566ca02c3cea7765410c7c3757a"
server_key_iv: "dacf171941e5ae41f93b12f1635c2250"
server_key_version: "encryptionkey@0"
I20260811 10:11:43.689370 14940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:43.701261 14940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:43.703878 14940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:43.704083 14940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:43.708837 14940 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:40623
I20260811 10:11:43.708873 15015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:40623 every 8 connection(s)
I20260811 10:11:43.709268 14940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:43.710352 15016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:43.712927 15016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456: Bootstrap starting.
I20260811 10:11:43.713551 15016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:43.713914 15016 log.cc:824] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:43.714820 15016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456: No bootstrap required, opened a new log
I20260811 10:11:43.715293 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 14940
I20260811 10:11:43.715437 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:43.715646 23236 external_mini_cluster.cc:1428] Setting key 5ec05f4ce028e9e48d5c7e3aede95f50
I20260811 10:11:43.717248 15016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ba079bfe9d4436aae43ce97d0a1456" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40623 } }
I20260811 10:11:43.717423 15016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:43.717447 15016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ba079bfe9d4436aae43ce97d0a1456, State: Initialized, Role: FOLLOWER
I20260811 10:11:43.717651 15016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ba079bfe9d4436aae43ce97d0a1456" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40623 } }
I20260811 10:11:43.717732 15016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:43.717756 15016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:43.717814 15016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:43.718446 15016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ba079bfe9d4436aae43ce97d0a1456" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40623 } }
I20260811 10:11:43.718539 15016 leader_election.cc:302] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ba079bfe9d4436aae43ce97d0a1456; no voters:
I20260811 10:11:43.718780 15016 leader_election.cc:288] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:43.718814 15021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:43.718992 15021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [term 1 LEADER]: Becoming Leader. State: Replica: 13ba079bfe9d4436aae43ce97d0a1456, State: Running, Role: LEADER
I20260811 10:11:43.719082 15016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:43.719139 15021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ba079bfe9d4436aae43ce97d0a1456" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40623 } }
I20260811 10:11:43.719558 15021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ba079bfe9d4436aae43ce97d0a1456. Latest consensus state: current_term: 1 leader_uuid: "13ba079bfe9d4436aae43ce97d0a1456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ba079bfe9d4436aae43ce97d0a1456" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40623 } } }
I20260811 10:11:43.719686 15023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13ba079bfe9d4436aae43ce97d0a1456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ba079bfe9d4436aae43ce97d0a1456" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 40623 } } }
I20260811 10:11:43.719796 15023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:43.719703 15021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:43.720211 15032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:43.720825 15032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:43.723091 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:43.716250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55531 (local address 127.22.177.62:40623)
0811 10:11:43.716449 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:43.716457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:43.716486 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:11:43.717251 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:43.717266 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:43.717342 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:43.717550 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:43.717875 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:43.719453 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:43.720638 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:43.721353 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:43.721486 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:43.722433 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:43.722437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:43.722437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:43.722619 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:43.722651 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:43.722662 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:43.722763 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:43.722850 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 10:11:43.722885 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260811 10:11:43.724278 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:43.726068 15032 catalog_manager.cc:1347] Generated new cluster ID: 32bf4a95f9394a91a6a7df4ba242c3fc
I20260811 10:11:43.726137 15032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:43.728945 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:43.725795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60203 (local address 127.22.177.62:40623)
0811 10:11:43.725838 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:43.725840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:43.725965 (+ 125us) server_negotiation.cc:407] Connection header received
0811 10:11:43.726039 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:43.726040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:43.726054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:43.726086 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:43.726311 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:43.726798 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:43.727548 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:43.727809 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:43.727887 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:43.728657 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:43.728658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:43.728659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:43.728664 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:43.728681 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:43.728686 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:43.728745 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:43.728829 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 10:11:43.728843 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:11:43.736824 15032 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:43.737609 15032 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:43.743717 15032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13ba079bfe9d4436aae43ce97d0a1456: Generated new TSK 0
I20260811 10:11:43.743964 15032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:43.748610 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 14940
W20260811 10:11:43.756567 14924 connection.cc:535] client connection to 127.22.177.62:40623 recv error: Network error: failed to read from TLS socket (remote: 127.22.177.62:40623): Connection reset by peer (error 104)
2026-08-11T10:11:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:43 dist-test-slave-g350 krb5kdc[15051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:43 dist-test-slave-g350 krb5kdc[15051](info): set up 2 sockets
Aug 11 10:11:43 dist-test-slave-g350 krb5kdc[15051](info): commencing operation
krb5kdc: starting...
W20260811 10:11:44.875118 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:44 dist-test-slave-g350 krb5kdc[15051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:45.022856 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:34547
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:34547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:45.111002 15066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:45.111279 15066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:45.111335 15066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:45.112613 15066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:45.112708 15066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:45.112735 15066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:45.112756 15066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:45.112787 15066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:45.114472 15066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:34547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:34547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:45.114799 15066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:45.115062 15066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:45.120661 15072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:45.120700 15071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:45.120702 15066 server_base.cc:1034] running on GCE node
W20260811 10:11:45.120671 15074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:45.121203 15066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:45.121577 15066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:45.122821 15066 hybrid_clock.cc:648] HybridClock initialized: now 1786443105122811 us; error 55 us; skew 500 ppm
Aug 11 10:11:45 dist-test-slave-g350 krb5kdc[15051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443105, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:45.125772 15066 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:45.126295 15066 webserver.cc:460] Webserver started at http://127.22.177.62:38319/ using document root <none> and password file <none>
I20260811 10:11:45.126560 15066 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:45.126634 15066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:45.126783 15066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:45.127945 15066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "13d965b7681a47e3afe429d9c828d30e"
format_stamp: "Formatted at 2026-08-11 10:11:45 on dist-test-slave-g350"
server_key: "958319dff0b14f39ec6e10597c12afee"
server_key_iv: "f090b5ff2021a536ea3892558ff8f051"
server_key_version: "encryptionkey@0"
I20260811 10:11:45.128312 15066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "13d965b7681a47e3afe429d9c828d30e"
format_stamp: "Formatted at 2026-08-11 10:11:45 on dist-test-slave-g350"
server_key: "958319dff0b14f39ec6e10597c12afee"
server_key_iv: "f090b5ff2021a536ea3892558ff8f051"
server_key_version: "encryptionkey@0"
I20260811 10:11:45.129832 15066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:11:45.130749 15081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:45.130995 15066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:45.131114 15066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "13d965b7681a47e3afe429d9c828d30e"
format_stamp: "Formatted at 2026-08-11 10:11:45 on dist-test-slave-g350"
server_key: "958319dff0b14f39ec6e10597c12afee"
server_key_iv: "f090b5ff2021a536ea3892558ff8f051"
server_key_version: "encryptionkey@0"
I20260811 10:11:45.131194 15066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:45.134799 15066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:45.137101 15066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:45.137305 15066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:45.141759 15066 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:34547
I20260811 10:11:45.141846 15142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:34547 every 8 connection(s)
I20260811 10:11:45.142195 15066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:45.143285 15143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:45.145583 15143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e: Bootstrap starting.
I20260811 10:11:45.146353 15143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:45.146644 15143 log.cc:824] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:45.147450 15143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e: No bootstrap required, opened a new log
I20260811 10:11:45.149111 15143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d965b7681a47e3afe429d9c828d30e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34547 } }
I20260811 10:11:45.149253 15143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:45.149276 15143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d965b7681a47e3afe429d9c828d30e, State: Initialized, Role: FOLLOWER
I20260811 10:11:45.149442 15143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d965b7681a47e3afe429d9c828d30e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34547 } }
I20260811 10:11:45.149511 15143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:45.149534 15143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:45.149632 15143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:45.149847 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15066
I20260811 10:11:45.149996 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:45.150197 23236 external_mini_cluster.cc:1428] Setting key bfa933f5da9b6513c6443a73563885c4
I20260811 10:11:45.150871 15143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d965b7681a47e3afe429d9c828d30e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34547 } }
I20260811 10:11:45.151047 15143 leader_election.cc:302] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13d965b7681a47e3afe429d9c828d30e; no voters:
I20260811 10:11:45.151222 15143 leader_election.cc:288] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:45.151330 15148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:45.151491 15143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:45.151510 15148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [term 1 LEADER]: Becoming Leader. State: Replica: 13d965b7681a47e3afe429d9c828d30e, State: Running, Role: LEADER
I20260811 10:11:45.151696 15148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d965b7681a47e3afe429d9c828d30e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34547 } }
I20260811 10:11:45.152076 15149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13d965b7681a47e3afe429d9c828d30e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d965b7681a47e3afe429d9c828d30e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34547 } } }
I20260811 10:11:45.152169 15149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:45.152451 15150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13d965b7681a47e3afe429d9c828d30e. Latest consensus state: current_term: 1 leader_uuid: "13d965b7681a47e3afe429d9c828d30e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d965b7681a47e3afe429d9c828d30e" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 34547 } } }
I20260811 10:11:45.152757 15150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:45.153102 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:45.150783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43687 (local address 127.22.177.62:34547)
0811 10:11:45.151024 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:45.151034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:45.151071 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:11:45.151197 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:45.151206 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:45.151249 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:45.151945 (+ 696us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:45.152512 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:45.152516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:45.152516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:45.152519 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:45.152552 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:45.152565 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:45.152646 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:45.152840 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 10:11:45.152886 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":113,"threads_started":1}
W20260811 10:11:45.153455 15165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:45.153566 15165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:45.153674 15157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:45.153826 15157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:45.154223 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:45.155436 15157 catalog_manager.cc:1347] Generated new cluster ID: 50b71e26b3eb41d4b9198a1877673b52
I20260811 10:11:45.155491 15157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:45.156915 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:45.155796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34599 (local address 127.22.177.62:34547)
0811 10:11:45.155852 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:45.155854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:45.155955 (+ 101us) server_negotiation.cc:407] Connection header received
0811 10:11:45.156015 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:45.156016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:45.156020 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:45.156074 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:45.156697 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:45.156698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:45.156699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:45.156702 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:45.156723 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:45.156727 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:45.156778 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:45.156845 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 10:11:45.156857 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:11:45.163516 15157 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:45.163748 15157 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:45.171600 15157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13d965b7681a47e3afe429d9c828d30e: Generated new TSK 0
I20260811 10:11:45.171855 15157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:45.176494 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15066
2026-08-11T10:11:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:45 dist-test-slave-g350 krb5kdc[15177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:45 dist-test-slave-g350 krb5kdc[15177](info): set up 2 sockets
Aug 11 10:11:45 dist-test-slave-g350 krb5kdc[15177](info): commencing operation
krb5kdc: starting...
W20260811 10:11:46.289651 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:46 dist-test-slave-g350 krb5kdc[15177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:46.433368 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35159
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:36073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:35159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:46.520618 15190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:46.520901 15190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:46.520943 15190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:46.522154 15190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:46.522236 15190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:46.522261 15190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:46.522291 15190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:46.522322 15190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:46.523881 15190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:35159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:35159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:46.524186 15190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:46.524441 15190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:46.529847 15199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:46.529838 15196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:46.529841 15197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:46.530133 15190 server_base.cc:1034] running on GCE node
I20260811 10:11:46.530592 15190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:46.530972 15190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:46.532151 15190 hybrid_clock.cc:648] HybridClock initialized: now 1786443106532101 us; error 64 us; skew 500 ppm
Aug 11 10:11:46 dist-test-slave-g350 krb5kdc[15177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443106, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:46.534981 15190 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:46.535555 15190 webserver.cc:460] Webserver started at http://127.22.177.62:41537/ using document root <none> and password file <none>
I20260811 10:11:46.535802 15190 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:46.535903 15190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:46.536078 15190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:46.537221 15190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "3e39b53a12f04835908fa2df8d055466"
format_stamp: "Formatted at 2026-08-11 10:11:46 on dist-test-slave-g350"
server_key: "653bfe5daf78f1a2bcdc511a2842bb45"
server_key_iv: "20c723b3f762ce5a559afdf6ba28dab2"
server_key_version: "encryptionkey@0"
I20260811 10:11:46.537700 15190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "3e39b53a12f04835908fa2df8d055466"
format_stamp: "Formatted at 2026-08-11 10:11:46 on dist-test-slave-g350"
server_key: "653bfe5daf78f1a2bcdc511a2842bb45"
server_key_iv: "20c723b3f762ce5a559afdf6ba28dab2"
server_key_version: "encryptionkey@0"
I20260811 10:11:46.539187 15190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:46.540053 15206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:46.540261 15190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:46.540407 15190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "3e39b53a12f04835908fa2df8d055466"
format_stamp: "Formatted at 2026-08-11 10:11:46 on dist-test-slave-g350"
server_key: "653bfe5daf78f1a2bcdc511a2842bb45"
server_key_iv: "20c723b3f762ce5a559afdf6ba28dab2"
server_key_version: "encryptionkey@0"
I20260811 10:11:46.540488 15190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:46.558039 15190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:46.560832 15190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:46.561007 15190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:46.565580 15190 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:35159
I20260811 10:11:46.565680 15263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:35159 every 8 connection(s)
I20260811 10:11:46.566072 15190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:46.567001 15264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:46.569612 15264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466: Bootstrap starting.
I20260811 10:11:46.570142 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15190
I20260811 10:11:46.570212 15264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:46.570288 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:46.570499 23236 external_mini_cluster.cc:1428] Setting key 4f11d4778552db8896f67b300268916f
I20260811 10:11:46.570593 15264 log.cc:824] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:46.571444 15264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466: No bootstrap required, opened a new log
I20260811 10:11:46.573580 15264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e39b53a12f04835908fa2df8d055466" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35159 } }
I20260811 10:11:46.573834 15264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:46.573896 15264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e39b53a12f04835908fa2df8d055466, State: Initialized, Role: FOLLOWER
I20260811 10:11:46.574033 15264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e39b53a12f04835908fa2df8d055466" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35159 } }
I20260811 10:11:46.574119 15264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:46.574165 15264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:46.574214 15264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:46.574819 15264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e39b53a12f04835908fa2df8d055466" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35159 } }
I20260811 10:11:46.574959 15264 leader_election.cc:302] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e39b53a12f04835908fa2df8d055466; no voters:
I20260811 10:11:46.575158 15264 leader_election.cc:288] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:46.575204 15269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:46.575371 15269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [term 1 LEADER]: Becoming Leader. State: Replica: 3e39b53a12f04835908fa2df8d055466, State: Running, Role: LEADER
I20260811 10:11:46.575445 15264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:46.575562 15269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e39b53a12f04835908fa2df8d055466" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35159 } }
I20260811 10:11:46.576280 15270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e39b53a12f04835908fa2df8d055466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e39b53a12f04835908fa2df8d055466" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35159 } } }
I20260811 10:11:46.576297 15271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e39b53a12f04835908fa2df8d055466. Latest consensus state: current_term: 1 leader_uuid: "3e39b53a12f04835908fa2df8d055466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e39b53a12f04835908fa2df8d055466" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 35159 } } }
I20260811 10:11:46.576483 15270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:46.576494 15271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:46.576758 15279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:46.577315 15279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:11:46 dist-test-slave-g350 krb5kdc[15177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:46.580385 15279 catalog_manager.cc:1347] Generated new cluster ID: 7a2f0ebe7e3b44d9bc0c9330395ad33d
I20260811 10:11:46.580451 15279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:46.581868 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:46.571089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59771 (local address 127.22.177.62:35159)
0811 10:11:46.571271 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:46.571281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:46.571360 (+ 79us) server_negotiation.cc:407] Connection header received
0811 10:11:46.571853 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:46.571861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:46.571910 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:46.572058 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:46.572715 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:46.573687 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:46.574628 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:46.575162 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:46.575628 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:46.578306 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:46.578310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:46.578311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:46.578323 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:46.580130 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:46.580579 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:46.580583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:46.580584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:46.580687 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:46.580909 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:46.580910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:46.580910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:46.581160 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:46.581345 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:46.581447 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 10:11:46.581624 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260811 10:11:46.582890 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:46.590718 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:46.584308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55525 (local address 127.22.177.62:35159)
0811 10:11:46.584347 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:46.584349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:46.584470 (+ 121us) server_negotiation.cc:407] Connection header received
0811 10:11:46.584533 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:46.584534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:46.584544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:46.584598 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:46.585124 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:46.585724 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:46.586504 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:46.586729 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:46.586817 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:46.588089 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:46.588090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:46.588090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:46.588097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:46.589509 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:46.589901 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:46.589903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:46.589903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:46.589965 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:46.590170 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:46.590171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:46.590172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:46.590287 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:46.590394 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:46.590528 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 10:11:46.590609 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:11:46.599300 15279 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:46.599884 15279 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:46.604200 15279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e39b53a12f04835908fa2df8d055466: Generated new TSK 0
I20260811 10:11:46.604424 15279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:46.606945 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15190
2026-08-11T10:11:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:46 dist-test-slave-g350 krb5kdc[15302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:46 dist-test-slave-g350 krb5kdc[15302](info): set up 2 sockets
Aug 11 10:11:46 dist-test-slave-g350 krb5kdc[15302](info): commencing operation
krb5kdc: starting...
W20260811 10:11:47.728417 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:47 dist-test-slave-g350 krb5kdc[15302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:47.870674 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45831
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:45831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:47.958021 15316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:47.958282 15316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:47.958336 15316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:47.959542 15316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:47.959613 15316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:47.959640 15316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:47.959671 15316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:47.959707 15316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:47.961306 15316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:45831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:45831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:47.961650 15316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:47.961913 15316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:47.967487 15326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:47.967490 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:47.967491 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:47.967685 15316 server_base.cc:1034] running on GCE node
I20260811 10:11:47.967995 15316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:47.968330 15316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:47.969497 15316 hybrid_clock.cc:648] HybridClock initialized: now 1786443107969475 us; error 44 us; skew 500 ppm
Aug 11 10:11:47 dist-test-slave-g350 krb5kdc[15302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:47.977375 15316 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:47.978047 15316 webserver.cc:460] Webserver started at http://127.22.177.62:35743/ using document root <none> and password file <none>
I20260811 10:11:47.978322 15316 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:47.978404 15316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:47.978581 15316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:47.979799 15316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "23785da725a04bf491241d792f128ba8"
format_stamp: "Formatted at 2026-08-11 10:11:47 on dist-test-slave-g350"
server_key: "bc728f052f76f1b9ac5c73a8107d008a"
server_key_iv: "4513b8e4d36dbaa1548de137da64d689"
server_key_version: "encryptionkey@0"
I20260811 10:11:47.980237 15316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "23785da725a04bf491241d792f128ba8"
format_stamp: "Formatted at 2026-08-11 10:11:47 on dist-test-slave-g350"
server_key: "bc728f052f76f1b9ac5c73a8107d008a"
server_key_iv: "4513b8e4d36dbaa1548de137da64d689"
server_key_version: "encryptionkey@0"
I20260811 10:11:47.981920 15316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:47.982774 15333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:47.983062 15316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:47.983270 15316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "23785da725a04bf491241d792f128ba8"
format_stamp: "Formatted at 2026-08-11 10:11:47 on dist-test-slave-g350"
server_key: "bc728f052f76f1b9ac5c73a8107d008a"
server_key_iv: "4513b8e4d36dbaa1548de137da64d689"
server_key_version: "encryptionkey@0"
I20260811 10:11:47.983364 15316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:47.991951 15316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:47.994307 15316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:47.994488 15316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:47.999071 15316 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:45831
I20260811 10:11:47.999133 15391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:45831 every 8 connection(s)
I20260811 10:11:47.999524 15316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:48.000461 15392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:48.002980 15392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8: Bootstrap starting.
I20260811 10:11:48.003633 15392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:48.003943 15392 log.cc:824] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:48.004832 15392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8: No bootstrap required, opened a new log
I20260811 10:11:48.006697 15392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23785da725a04bf491241d792f128ba8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45831 } }
I20260811 10:11:48.006860 15392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:48.006886 15392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23785da725a04bf491241d792f128ba8, State: Initialized, Role: FOLLOWER
I20260811 10:11:48.007058 15392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23785da725a04bf491241d792f128ba8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45831 } }
I20260811 10:11:48.007149 15392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:48.007200 15392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:48.007272 15392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:48.007261 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15316
I20260811 10:11:48.007416 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:48.007647 23236 external_mini_cluster.cc:1428] Setting key 9658a52f055cdb93867659823a572aa0
I20260811 10:11:48.008575 15392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23785da725a04bf491241d792f128ba8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45831 } }
I20260811 10:11:48.008699 15392 leader_election.cc:302] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23785da725a04bf491241d792f128ba8; no voters:
I20260811 10:11:48.008858 15392 leader_election.cc:288] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:48.008920 15398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:48.009130 15392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:48.009145 15398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 23785da725a04bf491241d792f128ba8, State: Running, Role: LEADER
I20260811 10:11:48.009351 15398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23785da725a04bf491241d792f128ba8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45831 } }
I20260811 10:11:48.009804 15399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23785da725a04bf491241d792f128ba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23785da725a04bf491241d792f128ba8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45831 } } }
I20260811 10:11:48.009821 15400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23785da725a04bf491241d792f128ba8. Latest consensus state: current_term: 1 leader_uuid: "23785da725a04bf491241d792f128ba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23785da725a04bf491241d792f128ba8" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 45831 } } }
I20260811 10:11:48.010003 15399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:48.010015 15400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:48.010879 15407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:48.011727 15407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:48.014461 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:48.008338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34231 (local address 127.22.177.62:45831)
0811 10:11:48.008609 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:48.008619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:48.008658 (+ 39us) server_negotiation.cc:407] Connection header received
0811 10:11:48.008782 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:48.008790 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:48.008868 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:48.009026 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:48.009312 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:48.011005 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:48.012160 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:48.012815 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:48.012941 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:48.013718 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:48.013723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:48.013724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:48.013975 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:48.014014 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:48.014030 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:48.014106 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:48.014168 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 10:11:48.014243 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":139,"threads_started":1}
I20260811 10:11:48.015573 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:48.020292 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:48.017106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60177 (local address 127.22.177.62:45831)
0811 10:11:48.017173 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:48.017175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:48.017224 (+ 49us) server_negotiation.cc:407] Connection header received
0811 10:11:48.017291 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:48.017292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:48.017301 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:48.017352 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:48.017482 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:48.018031 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:48.019130 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:48.019320 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:48.019371 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:48.020043 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:48.020044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:48.020044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:48.020049 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:48.020063 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:48.020067 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:48.020110 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:48.020167 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 10:11:48.020187 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:11:48.022112 15407 catalog_manager.cc:1347] Generated new cluster ID: 3fad7630d306400daf8c352754447d1a
I20260811 10:11:48.022177 15407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:48.052672 15407 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:48.053289 15407 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:48.061582 15407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23785da725a04bf491241d792f128ba8: Generated new TSK 0
I20260811 10:11:48.061854 15407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:48.073457 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15316
2026-08-11T10:11:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:48 dist-test-slave-g350 krb5kdc[15426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:48 dist-test-slave-g350 krb5kdc[15426](info): set up 2 sockets
Aug 11 10:11:48 dist-test-slave-g350 krb5kdc[15426](info): commencing operation
krb5kdc: starting...
W20260811 10:11:49.180184 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:49 dist-test-slave-g350 krb5kdc[15426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:49.316875 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42479
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:49.403038 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:49.403260 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:49.403285 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:49.404455 15441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:49.404501 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:49.404516 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:49.404528 15441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:49.404541 15441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:49.406061 15441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:42479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:49.406275 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:49.406458 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:49.411644 15449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:49.411687 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:49.411644 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:49.411911 15441 server_base.cc:1034] running on GCE node
I20260811 10:11:49.412173 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:49.412508 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:49.413738 15441 hybrid_clock.cc:648] HybridClock initialized: now 1786443109413701 us; error 59 us; skew 500 ppm
Aug 11 10:11:49 dist-test-slave-g350 krb5kdc[15426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443109, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:49.416471 15441 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:49.416949 15441 webserver.cc:460] Webserver started at http://127.22.177.62:40611/ using document root <none> and password file <none>
I20260811 10:11:49.417212 15441 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:49.417284 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:49.417440 15441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:49.418632 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae"
format_stamp: "Formatted at 2026-08-11 10:11:49 on dist-test-slave-g350"
server_key: "8f9759d2ccb2688b6ae480c605a778f1"
server_key_iv: "5ce88cf7ddb8d4c060e4034bb19dd5aa"
server_key_version: "encryptionkey@0"
I20260811 10:11:49.419019 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae"
format_stamp: "Formatted at 2026-08-11 10:11:49 on dist-test-slave-g350"
server_key: "8f9759d2ccb2688b6ae480c605a778f1"
server_key_iv: "5ce88cf7ddb8d4c060e4034bb19dd5aa"
server_key_version: "encryptionkey@0"
I20260811 10:11:49.420454 15441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:49.421257 15457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:49.421518 15441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:49.421665 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae"
format_stamp: "Formatted at 2026-08-11 10:11:49 on dist-test-slave-g350"
server_key: "8f9759d2ccb2688b6ae480c605a778f1"
server_key_iv: "5ce88cf7ddb8d4c060e4034bb19dd5aa"
server_key_version: "encryptionkey@0"
I20260811 10:11:49.421753 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:49.425330 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:49.427397 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:49.427580 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:49.432013 15441 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:42479
I20260811 10:11:49.432044 15517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:42479 every 8 connection(s)
I20260811 10:11:49.432444 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:49.432756 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15441
I20260811 10:11:49.432898 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:49.433120 23236 external_mini_cluster.cc:1428] Setting key a5bd73f8e69842a140ceaaec2f8d52db
I20260811 10:11:49.433696 15518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:49.435806 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:49.433695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36479 (local address 127.22.177.62:42479)
0811 10:11:49.433918 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:49.433926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:49.433951 (+ 25us) server_negotiation.cc:407] Connection header received
0811 10:11:49.434429 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:49.434436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:49.434463 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:49.434580 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:49.435296 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:49.435300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:49.435300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:49.435304 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:49.435359 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:49.435370 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:49.435427 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:49.435536 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 10:11:49.435599 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260811 10:11:49.436903 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:49.437348 15518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae: Bootstrap starting.
I20260811 10:11:49.438011 15518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:49.438410 15518 log.cc:824] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:49.439241 15518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae: No bootstrap required, opened a new log
I20260811 10:11:49.439689 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:49.438538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59187 (local address 127.22.177.62:42479)
0811 10:11:49.438577 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:49.438579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:49.438721 (+ 142us) server_negotiation.cc:407] Connection header received
0811 10:11:49.438777 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:49.438778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:49.438782 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:49.438836 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:49.439485 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:49.439486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:49.439486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:11:49.439491 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:49.439506 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:11:49.439510 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:49.439573 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:49.439616 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 10:11:49.439629 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:11:49.441365 15518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42479 } }
I20260811 10:11:49.441506 15518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:11:49.441529 15518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a7a4cb6e79428b8d0b9df4b2eb20ae, State: Initialized, Role: FOLLOWER
I20260811 10:11:49.441672 15518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42479 } }
I20260811 10:11:49.441763 15518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:11:49.441788 15518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:11:49.441819 15518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:49.442350 15518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42479 } }
I20260811 10:11:49.442438 15518 leader_election.cc:302] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50a7a4cb6e79428b8d0b9df4b2eb20ae; no voters:
I20260811 10:11:49.442636 15518 leader_election.cc:288] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:11:49.442706 15530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:49.442900 15518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:49.442905 15530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [term 1 LEADER]: Becoming Leader. State: Replica: 50a7a4cb6e79428b8d0b9df4b2eb20ae, State: Running, Role: LEADER
I20260811 10:11:49.443027 15530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42479 } }
I20260811 10:11:49.443388 15530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50a7a4cb6e79428b8d0b9df4b2eb20ae. Latest consensus state: current_term: 1 leader_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42479 } } }
I20260811 10:11:49.443480 15530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:49.443403 15532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a7a4cb6e79428b8d0b9df4b2eb20ae" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42479 } } }
I20260811 10:11:49.443531 15532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:49.443771 15538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:49.444370 15538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:49.446035 15538 catalog_manager.cc:1347] Generated new cluster ID: 2c3f0d634f9e4188a84b5566a58f0a28
I20260811 10:11:49.446091 15538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:49.453194 15538 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:49.453392 15538 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:49.457984 15538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50a7a4cb6e79428b8d0b9df4b2eb20ae: Generated new TSK 0
I20260811 10:11:49.458199 15538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:11:49.465942 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15441
2026-08-11T10:11:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1393 ms)
[----------] 18 tests from AuthTokenIssuingTest (25971 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:49 dist-test-slave-g350 krb5kdc[15553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:49 dist-test-slave-g350 krb5kdc[15553](info): set up 2 sockets
Aug 11 10:11:49 dist-test-slave-g350 krb5kdc[15553](info): commencing operation
krb5kdc: starting...
W20260811 10:11:50.599303 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:50 dist-test-slave-g350 krb5kdc[15553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:50.733011 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42871
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:42871,127.22.177.61:33807,127.22.177.60:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:50.817499 15567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:11:50.817768 15567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:50.817786 15567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:50.817807 15567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:50.818917 15567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:50.818954 15567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:50.818969 15567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:50.818981 15567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:50.818993 15567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:50.820446 15567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:42871,127.22.177.61:33807,127.22.177.60:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:50.820669 15567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:50.820863 15567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:50.826025 15574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:50.826030 15577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:50.826030 15575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:50.826318 15567 server_base.cc:1034] running on GCE node
I20260811 10:11:50.826691 15567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:50.826974 15567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:50.828123 15567 hybrid_clock.cc:648] HybridClock initialized: now 1786443110828081 us; error 62 us; skew 500 ppm
Aug 11 10:11:50 dist-test-slave-g350 krb5kdc[15553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:50.831128 15567 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:50.831593 15567 webserver.cc:460] Webserver started at http://127.22.177.62:44537/ using document root <none> and password file <none>
I20260811 10:11:50.831809 15567 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:50.831872 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:50.831969 15567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:50.833140 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83"
format_stamp: "Formatted at 2026-08-11 10:11:50 on dist-test-slave-g350"
server_key: "720ca09a0b10b324cbdaf57eb0fca45a"
server_key_iv: "9bd368360f7eff7588821eaf43548589"
server_key_version: "encryptionkey@0"
I20260811 10:11:50.833455 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83"
format_stamp: "Formatted at 2026-08-11 10:11:50 on dist-test-slave-g350"
server_key: "720ca09a0b10b324cbdaf57eb0fca45a"
server_key_iv: "9bd368360f7eff7588821eaf43548589"
server_key_version: "encryptionkey@0"
I20260811 10:11:50.834952 15567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:50.835677 15585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:50.835891 15567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:11:50.835985 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83"
format_stamp: "Formatted at 2026-08-11 10:11:50 on dist-test-slave-g350"
server_key: "720ca09a0b10b324cbdaf57eb0fca45a"
server_key_iv: "9bd368360f7eff7588821eaf43548589"
server_key_version: "encryptionkey@0"
I20260811 10:11:50.836045 15567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:50.851428 15567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:50.853746 15567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:50.853876 15567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:50.858717 15567 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:42871
I20260811 10:11:50.858780 15649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:42871 every 8 connection(s)
I20260811 10:11:50.859225 15567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:50.860347 15650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:50.862092 15650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:11:50 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:50.869338 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15567
I20260811 10:11:50.869493 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:50.869726 23236 external_mini_cluster.cc:1428] Setting key 58268ab0213a990ee1f0df549ad68e70
I20260811 10:11:50.872167 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.862821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.22.177.62:42871)
0811 10:11:50.863534 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:50.863537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:50.863548 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:11:50.863605 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:50.863606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:50.863628 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:50.863690 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:50.864247 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:50.865105 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:50.866192 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:50.866506 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:50.866587 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:50.868720 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:50.868728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:50.868728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:50.868734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:50.870454 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:50.870937 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:50.870940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:50.870940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:50.871020 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:50.871267 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:50.871268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:50.871269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:50.871491 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:50.871590 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:50.871778 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 10:11:50.871917 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":83,"threads_started":1}
I20260811 10:11:50.872165 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.862770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42871 (local address 127.0.0.1:36922)
0811 10:11:50.862927 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.862974 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:50.863409 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:50.863730 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:50.863732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:50.863986 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:50.864145 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:50.864149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:50.865232 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:50.865233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:50.866094 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:50.866097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:50.866619 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:50.866620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:50.866680 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:50.867215 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:50.867223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:50.868631 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:50.870544 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:50.870552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:50.870554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:50.870847 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:50.871084 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:50.871085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:50.871085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:50.871202 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:50.871569 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:50.871575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:50.871646 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 10:11:50.871747 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 10:11:50.871982 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":91,"thread_start_us":79,"threads_started":1}
I20260811 10:11:50.873020 15650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:50.873715 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.873476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35152)
0811 10:11:50.873532 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.873655 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 10:11:50.873898 15588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
I20260811 10:11:50.874343 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.874270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35162)
0811 10:11:50.874300 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.874325 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260811 10:11:50.874475 15650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:33807: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.177.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab.
Entry for principal kudu/127.22.177.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab.
I20260811 10:11:50.901292 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:33807
--webserver_interface=127.22.177.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab
--principal=kudu/127.22.177.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:42871,127.22.177.61:33807,127.22.177.60:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:11:50.923727 15650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } attempt: 1
I20260811 10:11:50.924371 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.924188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35176)
0811 10:11:50.924262 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.924306 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 10:11:50.924840 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.924693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35188)
0811 10:11:50.924772 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.924810 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 10:11:50.925005 15650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:33807: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
W20260811 10:11:50.987564 15657 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:11:50.987792 15657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:50.987810 15657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:50.987831 15657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:50.988943 15657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:50.988984 15657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:50.988998 15657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:50.989012 15657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:50.989023 15657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:50.990525 15657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:42871,127.22.177.61:33807,127.22.177.60:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab
--principal=kudu/127.22.177.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:33807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.177.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:50.990737 15657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:50.990917 15657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:11:50.993170 15650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } attempt: 2
I20260811 10:11:50.993706 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.993513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35204)
0811 10:11:50.993614 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.993667 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 10:11:50.994115 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:50.994007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35206)
0811 10:11:50.994063 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:50.994094 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 10:11:50.994292 15650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:33807: Network error: Client connection negotiation failed: client connection to 127.22.177.61:33807: connect: Connection refused (error 111)
W20260811 10:11:50.996930 15662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:50.996937 15663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:50.997125 15657 server_base.cc:1034] running on GCE node
W20260811 10:11:50.996958 15665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:50.997514 15657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:50.997908 15657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:50.999087 15657 hybrid_clock.cc:648] HybridClock initialized: now 1786443110999076 us; error 44 us; skew 500 ppm
Aug 11 10:11:50 dist-test-slave-g350 krb5kdc[15553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:51.002041 15657 init.cc:377] Logged in from keytab as kudu/127.22.177.61@KRBTEST.COM (short username kudu)
I20260811 10:11:51.002573 15657 webserver.cc:460] Webserver started at http://127.22.177.61:41227/ using document root <none> and password file <none>
I20260811 10:11:51.002811 15657 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:51.002887 15657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:51.003046 15657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:51.004211 15657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data/instance:
uuid: "8f1105f6f6a84575a3fd18af180436d5"
format_stamp: "Formatted at 2026-08-11 10:11:50 on dist-test-slave-g350"
server_key: "39d4cb6643adc1c95068951c6f932da9"
server_key_iv: "51a043a0223c421f2789b3d92a1f4bb0"
server_key_version: "encryptionkey@0"
I20260811 10:11:51.004616 15657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal/instance:
uuid: "8f1105f6f6a84575a3fd18af180436d5"
format_stamp: "Formatted at 2026-08-11 10:11:50 on dist-test-slave-g350"
server_key: "39d4cb6643adc1c95068951c6f932da9"
server_key_iv: "51a043a0223c421f2789b3d92a1f4bb0"
server_key_version: "encryptionkey@0"
I20260811 10:11:51.006351 15657 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:11:51.007484 15672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:51.007776 15657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:51.007885 15657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal
uuid: "8f1105f6f6a84575a3fd18af180436d5"
format_stamp: "Formatted at 2026-08-11 10:11:50 on dist-test-slave-g350"
server_key: "39d4cb6643adc1c95068951c6f932da9"
server_key_iv: "51a043a0223c421f2789b3d92a1f4bb0"
server_key_version: "encryptionkey@0"
I20260811 10:11:51.007988 15657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:51.019150 15657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:51.021723 15657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:51.021901 15657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:51.027047 15657 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.61:33807
I20260811 10:11:51.027124 15732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.61:33807 every 8 connection(s)
I20260811 10:11:51.027482 15657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
I20260811 10:11:51.027791 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15657
I20260811 10:11:51.027933 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-1/wal/instance
I20260811 10:11:51.028106 23236 external_mini_cluster.cc:1428] Setting key 13fee14c6987ebe37a42bf3645b90783
I20260811 10:11:51.028553 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:51.030329 15733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.177.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.60@KRBTEST.COM" created.
I20260811 10:11:51.040871 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.031214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.22.177.62:42871)
0811 10:11:51.031292 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.031295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.031479 (+ 184us) server_negotiation.cc:407] Connection header received
0811 10:11:51.031978 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.031979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.031996 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.032077 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.032733 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.033569 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.034710 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.035089 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.035170 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.037857 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.037859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.037859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.037867 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.039389 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.039885 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.039886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.039886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.039943 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.040156 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.040156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.040157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.040294 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.040372 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.040663 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.040764 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:11:51.041050 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.031129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42871 (local address 127.0.0.1:36932)
0811 10:11:51.031343 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.031387 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.031884 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.032155 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.032159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.032428 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.032628 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.032632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.033700 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.033701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.034613 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.034617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.035224 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.035225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.035386 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.036056 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.036065 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.037740 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.039464 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.039472 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.039475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.039802 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.040000 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.040001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.040001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.040106 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.040363 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.040369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.040497 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.040611 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.040791 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
I20260811 10:11:51.043033 15733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:11:51.052037 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.043392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35214)
0811 10:11:51.043453 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.043466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.043516 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.043964 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.043965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.044205 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.044289 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.044292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.045281 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.045282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.046176 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.046179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.046670 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.046671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.046756 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.047182 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.047188 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.048767 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.050940 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.050941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.050942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.051114 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.051370 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.051371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.051371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.051433 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.051751 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.051752 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.051792 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.051865 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.051962 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 10:11:51.052146 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.043568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35214 (local address 127.22.177.61:33807)
0811 10:11:51.043769 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.043774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.043800 (+ 26us) server_negotiation.cc:407] Connection header received
0811 10:11:51.043827 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.043829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.043844 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.043942 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.044404 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.045179 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.046274 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.046591 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.046646 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.048872 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.048884 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.048884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.048899 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.050821 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.051214 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.051218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.051218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.051277 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.051508 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.051509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.051509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.051672 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.051764 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.051933 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.052017 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
I20260811 10:11:51.052841 15733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:51.053215 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.053088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34364)
0811 10:11:51.053147 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.053194 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 10:11:51.053401 15675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
I20260811 10:11:51.053872 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.053789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34374)
0811 10:11:51.053824 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.053851 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 10:11:51.053994 15733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:39587: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab.
Entry for principal kudu/127.22.177.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab.
I20260811 10:11:51.062047 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:39587
--webserver_interface=127.22.177.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab
--principal=kudu/127.22.177.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:42871,127.22.177.61:33807,127.22.177.60:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:11:51.085525 15650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } attempt: 3
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:11:51.093372 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.085999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35222)
0811 10:11:51.086066 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.086082 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.086146 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.086302 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.086303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.086491 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.086596 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.086599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.087275 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.087276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.087832 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.087842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.088230 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.088231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.088311 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.088858 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.088863 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.090461 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.092242 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.092244 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.092245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.092418 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.092644 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.092645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.092645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.092711 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.093009 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.093010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.093064 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.093156 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.093267 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 10:11:51.093372 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.086091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.22.177.61:33807)
0811 10:11:51.086164 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.086167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.086177 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:51.086207 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.086209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.086220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.086294 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.086666 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.087175 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.087930 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.088152 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.088230 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.090568 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.090570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.090570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.090579 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.092055 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.092500 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.092501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.092502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.092564 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.092777 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.092778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.092778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.092919 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.093018 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.093190 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.093267 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:11:51.093973 15650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:51.094460 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.094294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34388)
0811 10:11:51.094396 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.094433 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 10:11:51.094825 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.094725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34396)
0811 10:11:51.094776 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.094805 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 10:11:51.094964 15650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:39587: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
I20260811 10:11:51.103313 15733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } attempt: 1
I20260811 10:11:51.103904 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.103729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34398)
0811 10:11:51.103827 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.103869 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 10:11:51.104354 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.104224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34412)
0811 10:11:51.104297 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.104328 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 10:11:51.104506 15733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:39587: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
I20260811 10:11:51.126118 15650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } attempt: 1
I20260811 10:11:51.126672 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.126521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34414)
0811 10:11:51.126583 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.126634 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:11:51.127174 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.127010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34424)
0811 10:11:51.127066 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.127137 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 10:11:51.127305 15650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:39587: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
W20260811 10:11:51.150585 15740 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:11:51.150856 15740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:51.150900 15740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:51.150940 15740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:51.152144 15740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:51.152216 15740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:51.152240 15740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:51.152261 15740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:51.152289 15740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:51.153890 15740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:42871,127.22.177.61:33807,127.22.177.60:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab
--principal=kudu/127.22.177.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:39587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.177.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:51.154182 15740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:51.154422 15740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:51.160001 15747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:51.160007 15750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:51.160025 15740 server_base.cc:1034] running on GCE node
W20260811 10:11:51.160007 15748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:51.160522 15740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:51.160844 15740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:51.162009 15740 hybrid_clock.cc:648] HybridClock initialized: now 1786443111161985 us; error 42 us; skew 500 ppm
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443111, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:51.164588 15740 init.cc:377] Logged in from keytab as kudu/127.22.177.60@KRBTEST.COM (short username kudu)
I20260811 10:11:51.165066 15740 webserver.cc:460] Webserver started at http://127.22.177.60:41897/ using document root <none> and password file <none>
I20260811 10:11:51.165324 15740 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:51.165398 15740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:51.165549 15740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:51.166667 15740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data/instance:
uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d"
format_stamp: "Formatted at 2026-08-11 10:11:51 on dist-test-slave-g350"
server_key: "405e1ea4cabb04ea409c87e49a193d20"
server_key_iv: "f2dc8fb740aa108e2d26a5a21a1c85a6"
server_key_version: "encryptionkey@0"
I20260811 10:11:51.167037 15740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal/instance:
uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d"
format_stamp: "Formatted at 2026-08-11 10:11:51 on dist-test-slave-g350"
server_key: "405e1ea4cabb04ea409c87e49a193d20"
server_key_iv: "f2dc8fb740aa108e2d26a5a21a1c85a6"
server_key_version: "encryptionkey@0"
I20260811 10:11:51.168542 15740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:11:51.169306 15757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:51.169498 15740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:11:51.169628 15740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal
uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d"
format_stamp: "Formatted at 2026-08-11 10:11:51 on dist-test-slave-g350"
server_key: "405e1ea4cabb04ea409c87e49a193d20"
server_key_iv: "f2dc8fb740aa108e2d26a5a21a1c85a6"
server_key_version: "encryptionkey@0"
I20260811 10:11:51.169723 15740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:51.172673 15733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } attempt: 2
I20260811 10:11:51.173134 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.173013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34438)
0811 10:11:51.173061 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.173094 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:11:51.173496 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.173395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34450)
0811 10:11:51.173448 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.173473 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 10:11:51.173596 15733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:39587: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
I20260811 10:11:51.196162 15740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:51.198553 15740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:51.198712 15740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:51.202463 15650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } attempt: 2
I20260811 10:11:51.202961 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.202803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34456)
0811 10:11:51.202883 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.202918 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 10:11:51.203405 15740 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.60:39587
I20260811 10:11:51.203414 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.203258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34468)
0811 10:11:51.203330 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.203382 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 10:11:51.203444 15818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.60:39587 every 8 connection(s)
W20260811 10:11:51.203519 15650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:39587: Network error: Client connection negotiation failed: client connection to 127.22.177.60:39587: connect: Connection refused (error 111)
I20260811 10:11:51.203827 15740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
I20260811 10:11:51.204843 15819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:51.206677 15819 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:51.208515 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15740
I20260811 10:11:51.208608 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786442825609099-23236-0/minicluster-data/master-2/wal/instance
I20260811 10:11:51.208820 23236 external_mini_cluster.cc:1428] Setting key 6a74348ee0912ec06ab6adceb033170a
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443111, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:11:51.217186 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.207427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.22.177.62:42871)
0811 10:11:51.207508 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.207511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.207689 (+ 178us) server_negotiation.cc:407] Connection header received
0811 10:11:51.208125 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.208126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.208135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.208184 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.208889 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.209648 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.210968 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.211279 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.211345 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.213894 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.213895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.213895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.213902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.215489 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.216049 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.216051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.216051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.216114 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.216430 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.216431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.216431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.216585 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.216664 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.216986 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.217068 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 10:11:51.217496 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.207378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:42871 (local address 127.0.0.1:36936)
0811 10:11:51.207569 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.207607 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.208044 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.208242 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.208245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.208506 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.208794 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.208799 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.209746 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.209747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.210888 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.210891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.211396 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.211397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.211524 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.212126 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.212136 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.213805 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.215599 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.215607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.215609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.215973 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.216204 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.216205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.216206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.216360 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.216687 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.216695 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.216832 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.216941 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.217121 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":120,"threads_started":1}
I20260811 10:11:51.218570 15819 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:51.218824 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.209230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.22.177.60:39587)
0811 10:11:51.209421 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.209426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.209500 (+ 74us) server_negotiation.cc:407] Connection header received
0811 10:11:51.209529 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.209531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.209550 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.209662 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.210485 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.211463 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.212501 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.212786 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.212876 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.215300 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.215309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.215310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.215326 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.217244 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.217868 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.217872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.217872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.217939 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.218196 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.218197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.218198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.218385 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.218531 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.218604 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.218685 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":126,"threads_started":1}
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443111, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:51.225762 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.218857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:33807 (local address 127.0.0.1:35234)
0811 10:11:51.218968 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.218979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.219027 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.219390 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.219391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.219517 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.219573 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.219574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.220190 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.220191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.220672 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.220674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.221049 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.221050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.221110 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.221513 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.221518 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.223023 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.224676 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.224677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.224678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.224877 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.225113 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.225114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.225114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.225185 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.225482 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.225483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.225519 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.225570 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.225697 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 10:11:51.225836 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.218939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.22.177.61:33807)
0811 10:11:51.218981 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.218983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.219012 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:11:51.219257 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.219258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.219269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.219345 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.219642 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.220109 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.220757 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.220974 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.221052 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.223114 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.223115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.223116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.223123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.224582 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.224959 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.224960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.224961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.225013 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.225248 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.225249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.225249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.225411 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.225488 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.225642 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.225728 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:11:51.226668 15819 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:51.227290 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.219612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.22.177.62:42871)
0811 10:11:51.219650 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.219652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.219695 (+ 43us) server_negotiation.cc:407] Connection header received
0811 10:11:51.219765 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.219766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.219775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.219826 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.220290 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.220731 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.221388 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.221626 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.221686 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.224258 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.224259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.224260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.224268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.225929 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.226314 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.226315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.226316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.226380 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.226619 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.226619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.226620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.226736 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.226954 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.227074 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.227231 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443111, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:11:51.234068 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.227000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34480 (local address 127.22.177.60:39587)
0811 10:11:51.227041 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.227044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.227052 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:51.227109 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.227110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.227120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.227163 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.227447 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.228122 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.228834 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.229031 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.229082 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.231079 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.231080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.231081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.231086 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.232590 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.232960 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.232962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.232962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.233040 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.233390 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.233391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.233391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.233543 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.233658 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.233856 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.233985 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":25}
I20260811 10:11:51.234069 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.226967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34480)
0811 10:11:51.227008 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.227017 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.227055 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.227174 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.227175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.227306 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.227361 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.227362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.228197 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.228198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.228742 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.228743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.229100 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.229102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.229152 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.229559 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.229564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.230985 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.232665 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.232666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.232666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.232886 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.233241 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.233242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.233243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.233309 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.233655 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.233656 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.233705 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.233804 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.233921 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 10:11:51.235766 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.227940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.22.177.61:33807)
0811 10:11:51.227984 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.227986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.228026 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:11:51.228090 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.228091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.228098 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.228140 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.228596 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.229153 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.229927 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.230201 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.230255 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.232848 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.232849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.232850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.232858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.234685 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.235088 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.235089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.235089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.235138 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.235324 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.235325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.235325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.235436 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.235544 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.235652 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.235707 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:11:51.236037 15819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: Bootstrap starting.
I20260811 10:11:51.237159 15819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:51.237428 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:51.237532 15819 log.cc:824] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:51.238463 15819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: No bootstrap required, opened a new log
I20260811 10:11:51.238543 15616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36952
I20260811 10:11:51.240418 15819 raft_consensus.cc:348] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.240584 15819 raft_consensus.cc:729] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a59b7ba3e1ac480fb8a749a3fb35cf1d, State: Initialized, Role: FOLLOWER
I20260811 10:11:51.240689 15819 consensus_queue.cc:260] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.241039 15819 sys_catalog.cc:564] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:51.241725 15825 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.241853 15825 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:11:51.242971 15837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:51.243034 15837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:51.243423 15740 master_runner.cc:186] Error getting master registration for 127.22.177.62:42871: OK, instance_id { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" instance_seqno: 1786443110857368 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:11:51.245481 15740 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:11:51.264777 15733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } attempt: 3
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:11:51.273608 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.265205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34486)
0811 10:11:51.265287 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.265315 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.265364 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.265544 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.265544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.265771 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.266127 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.266130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.267085 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.267086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.267900 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.267902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.268387 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.268388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.268448 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.268987 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.268993 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.270344 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.272275 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.272276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.272276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.272562 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.272798 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.272799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.272800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.272871 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.273218 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.273218 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.273276 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.273352 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.273471 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 10:11:51.273633 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.265319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.22.177.60:39587)
0811 10:11:51.265366 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.265368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.265417 (+ 49us) server_negotiation.cc:407] Connection header received
0811 10:11:51.265447 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.265448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.265459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.265519 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.266234 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.266990 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.267986 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.268299 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.268377 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.270435 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.270437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.270438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.270446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.272182 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.272637 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.272638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.272639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.272704 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.272946 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.272948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.272948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.273144 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.273220 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.273393 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.273487 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:51.275424 15733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: Bootstrap starting.
I20260811 10:11:51.276301 15733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:51.276633 15733 log.cc:824] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:51.277657 15733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: No bootstrap required, opened a new log
I20260811 10:11:51.279795 15733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.279970 15733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1105f6f6a84575a3fd18af180436d5, State: Initialized, Role: FOLLOWER
I20260811 10:11:51.280073 15733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.280463 15733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:51.280433 15840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.280599 15840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:11:51.282449 15852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:51.282508 15852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:51.282680 15657 master_runner.cc:186] Error getting master registration for 127.22.177.62:42871: OK, instance_id { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" instance_seqno: 1786443110857368 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:11:51.285727 15657 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:11:51.302673 15650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } attempt: 3
Aug 11 10:11:51 dist-test-slave-g350 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443110, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:11:51.311666 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.303082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:39587 (local address 127.0.0.1:34500)
0811 10:11:51.303154 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:51.303166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:51.303210 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:51.303421 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:51.303422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:51.303627 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:51.303685 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.303688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.304632 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.304633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.305484 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:51.305488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.306094 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.306096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:51.306165 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.306718 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:51.306723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:51.308481 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:51.310473 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.310474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.310475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.310736 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.310961 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:51.310962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:51.310962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:51.311028 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:51.311313 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:51.311313 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:51.311368 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 10:11:51.311438 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:11:51.311548 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 10:11:51.311714 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.303170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34500 (local address 127.22.177.60:39587)
0811 10:11:51.303246 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.303248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.303277 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:11:51.303308 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.303309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.303321 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.303407 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.303779 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.304534 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.305613 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.305947 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.306061 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.308592 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.308594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.308594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.308602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.310380 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.310807 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.310809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.310809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.310885 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.311097 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.311097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.311098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.311247 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.311316 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.311520 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.311610 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 10:11:51.313988 15650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83: Bootstrap starting.
I20260811 10:11:51.314776 15650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:51.315107 15650 log.cc:824] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:51.316184 15650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83: No bootstrap required, opened a new log
I20260811 10:11:51.317734 15650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.317899 15650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d89aa6d0e4a46c3b342ac2d2040bb83, State: Initialized, Role: FOLLOWER
I20260811 10:11:51.317981 15650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.318312 15650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:51.318354 15616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 10:11:51.318430 15616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:51.319329 15616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.320252 15702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8f1105f6f6a84575a3fd18af180436d5"
I20260811 10:11:51.320399 15702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:51.321308 15702 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d89aa6d0e4a46c3b342ac2d2040bb83 in term 1.
I20260811 10:11:51.321740 15588 leader_election.cc:302] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d89aa6d0e4a46c3b342ac2d2040bb83, 8f1105f6f6a84575a3fd18af180436d5; no voters:
I20260811 10:11:51.322016 15791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d"
I20260811 10:11:51.322144 15791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [term 0 FOLLOWER]: Advancing to term 1
W20260811 10:11:51.322778 15866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:51.322839 15866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:51.322912 15791 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d89aa6d0e4a46c3b342ac2d2040bb83 in term 1.
I20260811 10:11:51.319600 15616 leader_election.cc:288] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [CANDIDATE]: Term 1 election: Requested vote from peers 8f1105f6f6a84575a3fd18af180436d5 (127.22.177.61:33807), a59b7ba3e1ac480fb8a749a3fb35cf1d (127.22.177.60:39587)
I20260811 10:11:51.324002 15567 master_runner.cc:263] No leader master found from master 2d89aa6d0e4a46c3b342ac2d2040bb83
I20260811 10:11:51.324044 15567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:11:51.318316 15855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.325397 15855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:51.325498 15855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:51.325614 15855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [term 1 LEADER]: Becoming Leader. State: Replica: 2d89aa6d0e4a46c3b342ac2d2040bb83, State: Running, Role: LEADER
I20260811 10:11:51.325749 15855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } }
I20260811 10:11:51.333120 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.324660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.22.177.62:42871)
0811 10:11:51.324701 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.324703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.324988 (+ 285us) server_negotiation.cc:407] Connection header received
0811 10:11:51.325094 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.325095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.325105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.325135 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.326048 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.326697 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.327738 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.328027 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.328073 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.329928 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.329930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.329931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.329939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.331710 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.332157 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.332158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.332158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.332222 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.332453 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.332454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.332454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.332593 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.332669 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.332922 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.333019 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:11:51.333858 15868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d89aa6d0e4a46c3b342ac2d2040bb83. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.333992 15868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:51.334240 15878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:51.334482 15878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:51.341811 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.325024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.22.177.60:39587)
0811 10:11:51.325061 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.325063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.334017 (+ 8954us) server_negotiation.cc:407] Connection header received
0811 10:11:51.334123 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.334124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.334135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.334168 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.334915 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.335593 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.336632 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.336938 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.336989 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.338614 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.338616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.338616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.338623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.340346 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.340770 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.340772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.340772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.340846 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.341093 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.341095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.341095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.341233 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.341327 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.341628 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.341714 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:11:51.344003 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:51.324705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.22.177.61:33807)
0811 10:11:51.324748 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.324750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.333580 (+ 8830us) server_negotiation.cc:407] Connection header received
0811 10:11:51.333747 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.333749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.333761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.333792 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.334814 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.335520 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.336516 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:51.336788 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:51.336846 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:51.340565 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:51.340567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:51.340567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:51.340575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:51.342338 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.343152 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.343153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.343153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.343210 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:51.343406 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:51.343407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:51.343408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:51.343536 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:51.343604 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:51.343814 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 10:11:51.343908 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:51.346542 15791 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [term 1 FOLLOWER]: Refusing update from remote peer 2d89aa6d0e4a46c3b342ac2d2040bb83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:51.346585 15702 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [term 1 FOLLOWER]: Refusing update from remote peer 2d89aa6d0e4a46c3b342ac2d2040bb83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:51.346928 15868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:51.347126 15855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:51.348467 15825 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d89aa6d0e4a46c3b342ac2d2040bb83. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.348564 15825 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:51.348626 15840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d89aa6d0e4a46c3b342ac2d2040bb83. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.348698 15840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:51.350378 15855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.350472 15855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:51.350615 15879 mvcc.cc:205] Tried to move back new op lower bound from 7317270984072568832 to 7317270983991279616. Current Snapshot: MvccSnapshot[applied={T|T < 7317270984072568832}]
I20260811 10:11:51.350955 15825 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.351058 15825 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:51.351330 15880 mvcc.cc:205] Tried to move back new op lower bound from 7317270984072568832 to 7317270983991279616. Current Snapshot: MvccSnapshot[applied={T|T < 7317270984072568832}]
I20260811 10:11:51.357452 15878 catalog_manager.cc:1347] Generated new cluster ID: f43df342ee5443af9db1de3816fdbef5
I20260811 10:11:51.357535 15878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:51.357867 15868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.357962 15868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:51.358317 15840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89aa6d0e4a46c3b342ac2d2040bb83" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 42871 } } peers { permanent_uuid: "8f1105f6f6a84575a3fd18af180436d5" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 33807 } } peers { permanent_uuid: "a59b7ba3e1ac480fb8a749a3fb35cf1d" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 39587 } } }
I20260811 10:11:51.358421 15840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:51.358888 15881 mvcc.cc:205] Tried to move back new op lower bound from 7317270984072568832 to 7317270983991279616. Current Snapshot: MvccSnapshot[applied={T|T < 7317270984072568832}]
I20260811 10:11:51.381240 15878 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:51.382061 15878 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:51.391443 15878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d89aa6d0e4a46c3b342ac2d2040bb83: Generated new TSK 0
I20260811 10:11:51.391822 15878 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:11:51.417527 15653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.415661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.22.177.62:42871)
0811 10:11:51.415711 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.415713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.416757 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 10:11:51.416874 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.416875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.416888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.416923 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.417449 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36982: BlockingRecv error: recv got EOF from 127.0.0.1:36982 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:11:51.418411 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.415950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.22.177.61:33807)
0811 10:11:51.415991 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.415993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.417826 (+ 1833us) server_negotiation.cc:407] Connection header received
0811 10:11:51.417929 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.417930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.417942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.417973 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.418344 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35252: BlockingRecv error: recv got EOF from 127.0.0.1:35252 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 10:11:51.419189 15823 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.416474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.22.177.60:39587)
0811 10:11:51.416515 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.416517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.418656 (+ 2139us) server_negotiation.cc:407] Connection header received
0811 10:11:51.418755 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.418756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.418767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.418796 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.419126 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34526: BlockingRecv error: recv got EOF from 127.0.0.1:34526 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 10:11:51.421814 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.420988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.22.177.61:33807)
0811 10:11:51.421030 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.421032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.421222 (+ 190us) server_negotiation.cc:407] Connection header received
0811 10:11:51.421342 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.421343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.421363 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.421392 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.421764 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35264: BlockingRecv error: recv got EOF from 127.0.0.1:35264 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 10:11:51.424896 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.424105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.22.177.61:33807)
0811 10:11:51.424140 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.424142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.424398 (+ 256us) server_negotiation.cc:407] Connection header received
0811 10:11:51.424494 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.424495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.424506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.424534 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.424850 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35274: BlockingRecv error: recv got EOF from 127.0.0.1:35274 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 10:11:51.428835 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.428067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35290 (local address 127.22.177.61:33807)
0811 10:11:51.428105 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.428107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.428348 (+ 241us) server_negotiation.cc:407] Connection header received
0811 10:11:51.428444 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.428445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.428456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.428484 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.428789 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35290: BlockingRecv error: recv got EOF from 127.0.0.1:35290 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 10:11:51.435134 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.434149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.22.177.61:33807)
0811 10:11:51.434197 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.434200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.434486 (+ 286us) server_negotiation.cc:407] Connection header received
0811 10:11:51.434596 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.434597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.434612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.434650 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.435064 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35302: BlockingRecv error: recv got EOF from 127.0.0.1:35302 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 10:11:51.446543 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.444528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.22.177.61:33807)
0811 10:11:51.444581 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.444583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.445911 (+ 1328us) server_negotiation.cc:407] Connection header received
0811 10:11:51.446001 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.446002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.446014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.446077 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.446465 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35308: BlockingRecv error: recv got EOF from 127.0.0.1:35308 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:11:51.464643 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.463837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.22.177.61:33807)
0811 10:11:51.463889 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.463892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.464042 (+ 150us) server_negotiation.cc:407] Connection header received
0811 10:11:51.464105 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.464106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.464118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.464196 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.464566 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35318: BlockingRecv error: recv got EOF from 127.0.0.1:35318 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:11:51.498770 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.498018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.22.177.61:33807)
0811 10:11:51.498085 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.498087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.498176 (+ 89us) server_negotiation.cc:407] Connection header received
0811 10:11:51.498248 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.498249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.498264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.498331 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.498696 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35334: BlockingRecv error: recv got EOF from 127.0.0.1:35334 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260811 10:11:51.564852 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.564066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.22.177.61:33807)
0811 10:11:51.564147 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.564149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.564230 (+ 81us) server_negotiation.cc:407] Connection header received
0811 10:11:51.564292 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.564293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.564305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.564351 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.564757 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35340: BlockingRecv error: recv got EOF from 127.0.0.1:35340 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 10:11:51.695021 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.694300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.22.177.61:33807)
0811 10:11:51.694404 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.694406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.694414 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:51.694439 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.694439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.694450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.694497 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.694952 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35342: BlockingRecv error: recv got EOF from 127.0.0.1:35342 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 10:11:51.953433 15738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:51.952761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.22.177.61:33807)
0811 10:11:51.952850 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:51.952852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:51.952862 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:51.952895 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:51.952896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:51.952906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:51.952978 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:51.953361 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35350: BlockingRecv error: recv got EOF from 127.0.0.1:35350 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 10:11:52.243499 15837 catalog_manager.cc:1259] Loaded cluster ID: f43df342ee5443af9db1de3816fdbef5
I20260811 10:11:52.243604 15837 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: loading cluster ID for follower catalog manager: success
I20260811 10:11:52.244450 15837 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: acquiring CA information for follower catalog manager: success
I20260811 10:11:52.244817 15837 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a59b7ba3e1ac480fb8a749a3fb35cf1d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:11:52.282989 15852 catalog_manager.cc:1259] Loaded cluster ID: f43df342ee5443af9db1de3816fdbef5
I20260811 10:11:52.283071 15852 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: loading cluster ID for follower catalog manager: success
I20260811 10:11:52.283993 15852 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: acquiring CA information for follower catalog manager: success
I20260811 10:11:52.284322 15852 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f1105f6f6a84575a3fd18af180436d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:11:52.470629 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:52.467479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.22.177.61:33807)
0811 10:11:52.467702 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:52.467704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:52.467733 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:11:52.467992 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:52.467993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:52.468050 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:52.468140 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:52.468375 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.468912 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.469822 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.470025 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.470101 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:52.470247 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:52.470366 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:52.470428 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:52.470466 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 10:11:52.470487 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":69,"threads_started":1}
I20260811 10:11:52.475746 15968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:52.472986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.22.177.60:39587)
0811 10:11:52.473122 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:52.473125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:52.473170 (+ 45us) server_negotiation.cc:407] Connection header received
0811 10:11:52.473309 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:52.473310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:52.473351 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:52.473432 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:52.473535 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.474081 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.474878 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.475093 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.475152 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:52.475352 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:52.475465 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:52.475540 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:52.475604 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 10:11:52.475624 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260811 10:11:52.481096 15968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:52.478219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.22.177.60:39587)
0811 10:11:52.478264 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:52.478265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:52.478407 (+ 142us) server_negotiation.cc:407] Connection header received
0811 10:11:52.478462 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:52.478463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:52.478475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:52.478524 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:52.478662 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.479447 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.480225 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.480523 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.480587 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:52.480804 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:52.480843 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:52.480905 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:52.480958 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 10:11:52.480981 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:11:52.481395 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:52.478011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.22.177.61:33807)
0811 10:11:52.478060 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:52.478062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:52.478217 (+ 155us) server_negotiation.cc:407] Connection header received
0811 10:11:52.478337 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:52.478341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:52.478362 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:52.478429 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:52.478603 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.479392 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.480473 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.480787 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.480865 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:52.481069 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:52.481109 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:52.481169 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:52.481297 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 10:11:52.481326 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:52.481460 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:52.478013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.22.177.62:42871)
0811 10:11:52.478227 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:52.478230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:52.478239 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:11:52.478290 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:52.478291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:52.478377 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:52.478493 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:52.478636 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.479164 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.480238 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:52.480616 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:52.480683 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:52.480929 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:52.481048 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:52.481287 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:52.481307 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 10:11:52.481337 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":63,"threads_started":1}
I20260811 10:11:52.483420 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15567
I20260811 10:11:52.492360 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15657
I20260811 10:11:52.500419 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15740
2026-08-11T10:11:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3035 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:11:52 dist-test-slave-g350 krb5kdc[15980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:11:52 dist-test-slave-g350 krb5kdc[15980](info): set up 2 sockets
Aug 11 10:11:52 dist-test-slave-g350 krb5kdc[15980](info): commencing operation
krb5kdc: starting...
W20260811 10:11:53.634938 23236 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/test-user.keytab.
Aug 11 10:11:53 dist-test-slave-g350 krb5kdc[15980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.177.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
Entry for principal kudu/127.22.177.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab.
I20260811 10:11:53.777540 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37989
--webserver_interface=127.22.177.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37989,127.22.177.61:38963,127.22.177.60:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:11:53.870967 15993 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:11:53.871275 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:53.871321 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:53.871371 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:53.872735 15993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:53.872819 15993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:53.872846 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:53.872879 15993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:53.872915 15993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:53.874711 15993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37989,127.22.177.61:38963,127.22.177.60:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.62.keytab
--principal=kudu/127.22.177.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.62:37989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.177.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:53.875051 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:53.875339 15993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:53.881007 16001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:53.881007 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:53.881254 15993 server_base.cc:1034] running on GCE node
W20260811 10:11:53.881079 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:53.881817 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:53.882195 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:53.883381 15993 hybrid_clock.cc:648] HybridClock initialized: now 1786443113883342 us; error 55 us; skew 500 ppm
Aug 11 10:11:53 dist-test-slave-g350 krb5kdc[15980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:53.886344 15993 init.cc:377] Logged in from keytab as kudu/127.22.177.62@KRBTEST.COM (short username kudu)
I20260811 10:11:53.886894 15993 webserver.cc:460] Webserver started at http://127.22.177.62:44579/ using document root <none> and password file <none>
I20260811 10:11:53.887161 15993 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:53.887213 15993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:53.887370 15993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:53.888626 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data/instance:
uuid: "15ad49ccfeb44beb9c18588b83618d41"
format_stamp: "Formatted at 2026-08-11 10:11:53 on dist-test-slave-g350"
server_key: "6f7866851fc09586666c8a61c011ca67"
server_key_iv: "794ea572ed3fa26ec5dba098822785ee"
server_key_version: "encryptionkey@0"
I20260811 10:11:53.889068 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal/instance:
uuid: "15ad49ccfeb44beb9c18588b83618d41"
format_stamp: "Formatted at 2026-08-11 10:11:53 on dist-test-slave-g350"
server_key: "6f7866851fc09586666c8a61c011ca67"
server_key_iv: "794ea572ed3fa26ec5dba098822785ee"
server_key_version: "encryptionkey@0"
I20260811 10:11:53.890847 15993 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 10:11:53.891834 16008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:53.892102 15993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:53.892243 15993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal
uuid: "15ad49ccfeb44beb9c18588b83618d41"
format_stamp: "Formatted at 2026-08-11 10:11:53 on dist-test-slave-g350"
server_key: "6f7866851fc09586666c8a61c011ca67"
server_key_iv: "794ea572ed3fa26ec5dba098822785ee"
server_key_version: "encryptionkey@0"
I20260811 10:11:53.892330 15993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:53.908075 15993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:53.910739 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:53.910969 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:53.916306 15993 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.62:37989
I20260811 10:11:53.916348 16064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.62:37989 every 8 connection(s)
I20260811 10:11:53.916761 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/data/info.pb
I20260811 10:11:53.917814 16066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:53.919452 16066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:53.925415 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 15993
I20260811 10:11:53.925555 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-0/wal/instance
I20260811 10:11:53.925786 23236 external_mini_cluster.cc:1428] Setting key 45524caf35eabfac4c46a04bea3be04d
Aug 11 10:11:53 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:53.931236 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:53.920361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.22.177.62:37989)
0811 10:11:53.920500 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:53.920505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:53.920539 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:11:53.921163 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:53.921165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:53.921183 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:53.921273 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:53.922008 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:53.922836 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:53.923794 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:53.924110 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:53.924195 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:53.926609 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:53.926618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:53.926618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:53.926626 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:53.928972 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:53.929503 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:53.929507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:53.929507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:53.929868 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:53.930147 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:53.930148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:53.930148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:53.930411 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:53.930480 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:53.930762 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 10:11:53.931031 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260811 10:11:53.931250 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:53.920180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37989 (local address 127.0.0.1:52638)
0811 10:11:53.920402 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:53.920445 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:53.920994 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:53.921321 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:53.921324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:53.921715 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:53.921898 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:53.921903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:53.922937 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:53.922938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:53.923708 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:53.923710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:53.924247 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:53.924248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:53.924318 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:53.924840 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:53.924849 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:53.926516 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:53.929073 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:53.929082 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:53.929084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:53.929398 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:53.929946 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:53.929947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:53.929947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:53.930074 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:53.930495 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:53.930504 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:53.930603 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 10:11:53.930720 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:11:53.930961 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260811 10:11:53.932412 16066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:53.932862 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:53.932697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55516)
0811 10:11:53.932762 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:53.932824 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 10:11:53.933023 16011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
I20260811 10:11:53.933523 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:53.933404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55528)
0811 10:11:53.933460 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:53.933496 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 10:11:53.933665 16066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:38963: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.177.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab.
Entry for principal kudu/127.22.177.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab.
I20260811 10:11:53.963052 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:38963
--webserver_interface=127.22.177.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab
--principal=kudu/127.22.177.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37989,127.22.177.61:38963,127.22.177.60:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:11:53.982939 16066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } attempt: 1
I20260811 10:11:53.983577 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:53.983407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55540)
0811 10:11:53.983486 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:53.983537 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 10:11:53.984083 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:53.983977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55554)
0811 10:11:53.984028 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:53.984060 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 10:11:53.984205 16066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:38963: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
I20260811 10:11:54.052420 16066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } attempt: 2
I20260811 10:11:54.053086 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.052902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55562)
0811 10:11:54.052980 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.053034 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 10:11:54.053766 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.053531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55566)
0811 10:11:54.053679 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.053740 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260811 10:11:54.054029 16066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.61:38963: Network error: Client connection negotiation failed: client connection to 127.22.177.61:38963: connect: Connection refused (error 111)
W20260811 10:11:54.059191 16074 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:11:54.059449 16074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:54.059471 16074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:54.059530 16074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:54.060897 16074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:54.060945 16074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:54.060961 16074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:54.060976 16074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:54.061019 16074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:54.062938 16074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37989,127.22.177.61:38963,127.22.177.60:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.61.keytab
--principal=kudu/127.22.177.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.61:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.177.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:54.063253 16074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:54.063541 16074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:11:54.069561 16083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:54.069597 16080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:54.069726 16081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:54.069792 16074 server_base.cc:1034] running on GCE node
I20260811 10:11:54.070220 16074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:54.070503 16074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:54.071709 16074 hybrid_clock.cc:648] HybridClock initialized: now 1786443114071694 us; error 42 us; skew 500 ppm
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:54.074875 16074 init.cc:377] Logged in from keytab as kudu/127.22.177.61@KRBTEST.COM (short username kudu)
I20260811 10:11:54.075491 16074 webserver.cc:460] Webserver started at http://127.22.177.61:36951/ using document root <none> and password file <none>
I20260811 10:11:54.075812 16074 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:54.075912 16074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:54.076094 16074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:54.077432 16074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data/instance:
uuid: "89ab76135ad44857ad281e03425a5048"
format_stamp: "Formatted at 2026-08-11 10:11:54 on dist-test-slave-g350"
server_key: "4f77e42165df5de89925d216ccdd3827"
server_key_iv: "6c2dbe834b3237119c524dd0754a73f7"
server_key_version: "encryptionkey@0"
I20260811 10:11:54.077965 16074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal/instance:
uuid: "89ab76135ad44857ad281e03425a5048"
format_stamp: "Formatted at 2026-08-11 10:11:54 on dist-test-slave-g350"
server_key: "4f77e42165df5de89925d216ccdd3827"
server_key_iv: "6c2dbe834b3237119c524dd0754a73f7"
server_key_version: "encryptionkey@0"
I20260811 10:11:54.079825 16074 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:11:54.080840 16090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:54.081130 16074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:54.081288 16074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal
uuid: "89ab76135ad44857ad281e03425a5048"
format_stamp: "Formatted at 2026-08-11 10:11:54 on dist-test-slave-g350"
server_key: "4f77e42165df5de89925d216ccdd3827"
server_key_iv: "6c2dbe834b3237119c524dd0754a73f7"
server_key_version: "encryptionkey@0"
I20260811 10:11:54.081398 16074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:54.104485 16074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:54.107738 16074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:54.107916 16074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:54.113780 16074 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.61:38963
I20260811 10:11:54.114308 16074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/data/info.pb
I20260811 10:11:54.115418 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:54.117259 16150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:54.117736 16149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.61:38963 every 8 connection(s)
I20260811 10:11:54.120440 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 16074
I20260811 10:11:54.120575 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-1/wal/instance
I20260811 10:11:54.120834 23236 external_mini_cluster.cc:1428] Setting key 655dce0b4ff577c2b30ff83ce6f7120d
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:54.131659 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.119734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52650 (local address 127.22.177.62:37989)
0811 10:11:54.119840 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.119843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.120129 (+ 286us) server_negotiation.cc:407] Connection header received
0811 10:11:54.120876 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.120879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.120900 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.120990 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.121877 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.122752 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.124624 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.124987 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.125073 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.127894 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.127896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.127897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.127907 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.129960 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.130450 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.130452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.130453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.130518 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.130768 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.130769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.130770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.130939 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.131025 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.131456 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.131556 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.22.177.60@KRBTEST.COM; defaulting to no policy
I20260811 10:11:54.131826 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.119721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37989 (local address 127.0.0.1:52650)
0811 10:11:54.119977 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.120036 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.120750 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.121079 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.121084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.121450 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.121764 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.121770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.122887 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.122888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.124486 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.124490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.125141 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.125142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.125276 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.126052 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.126062 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.127781 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.130071 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.130079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.130082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.130341 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.130585 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.130586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.130586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.130699 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.131054 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.131062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.131296 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.131406 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.131572 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":146,"threads_started":1}
I20260811 10:11:54.132606 16150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.177.60@KRBTEST.COM" created.
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:11:54.141729 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.133025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.22.177.61:38963)
0811 10:11:54.133212 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.133216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.133240 (+ 24us) server_negotiation.cc:407] Connection header received
0811 10:11:54.133309 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.133310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.133324 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.133410 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.133825 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.134628 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.135428 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.135782 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.135868 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.138210 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.138221 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.138222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.138232 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.140237 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.140695 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.140699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.140699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.140764 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.140987 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.140988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.140988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.141176 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.141299 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.141417 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.141541 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
I20260811 10:11:54.141808 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.132963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55572)
0811 10:11:54.133145 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.133156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.133255 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.133473 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.133474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.133666 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.133752 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.133754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.134717 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.134718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.135333 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.135336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.135887 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.135888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.135958 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.136463 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.136468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.138077 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.140387 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.140388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.140389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.140588 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.140843 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.140844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.140845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.140908 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.141264 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.141266 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.141309 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.141384 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.141647 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":105}
I20260811 10:11:54.142786 16150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:54.143249 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.143102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57638)
0811 10:11:54.143174 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.143227 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 10:11:54.143467 16093 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
I20260811 10:11:54.144001 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.143892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57644)
0811 10:11:54.143949 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.143976 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 10:11:54.144125 16150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:35775: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
I20260811 10:11:54.145220 16066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } attempt: 3
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:11:54.153469 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.145689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55578 (local address 127.22.177.61:38963)
0811 10:11:54.145759 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.145762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.145864 (+ 102us) server_negotiation.cc:407] Connection header received
0811 10:11:54.145906 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.145907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.145918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.145973 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.146306 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.146901 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.147797 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.148062 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.148153 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.150593 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.150595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.150595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.150602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.152020 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.152463 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.152464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.152465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.152569 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.152887 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.152888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.152888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.153030 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.153096 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.153275 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.153397 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:11:54.153676 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.145583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55578)
0811 10:11:54.145744 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.145757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.145857 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.145991 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.145992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.146136 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.146210 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.146212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.146987 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.146988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.147697 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.147700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.148149 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.148150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.148217 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.148742 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.148748 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.150464 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.152119 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.152120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.152121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.152350 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.152711 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.152712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.152712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.152781 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.153120 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.153121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.153171 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.153276 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.153458 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 10:11:54.154186 16066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:54.154613 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.154485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57658)
0811 10:11:54.154551 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.154587 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 10:11:54.155166 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.155048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57670)
0811 10:11:54.155115 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.155149 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 10:11:54.155290 16066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:35775: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.177.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab.
Entry for principal kudu/127.22.177.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab.
I20260811 10:11:54.158411 23236 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wzoa9/build/release/bin/kudu
/tmp/dist-test-task2wzoa9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:35775
--webserver_interface=127.22.177.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab
--principal=kudu/127.22.177.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.177.20:40799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.177.62:37989,127.22.177.61:38963,127.22.177.60:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:11:54.186474 16066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } attempt: 1
I20260811 10:11:54.187011 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.186851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57682)
0811 10:11:54.186946 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.186980 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 10:11:54.187388 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.187292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57696)
0811 10:11:54.187338 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.187365 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 10:11:54.187498 16066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:35775: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
I20260811 10:11:54.193435 16150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } attempt: 1
I20260811 10:11:54.194084 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.193897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57698)
0811 10:11:54.193983 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.194042 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 10:11:54.194599 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.194478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57708)
0811 10:11:54.194534 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.194570 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 10:11:54.194774 16150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:35775: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
W20260811 10:11:54.255791 16157 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:11:54.256054 16157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:11:54.256081 16157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:11:54.256105 16157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:11:54.257249 16157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:11:54.257294 16157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:11:54.257311 16157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:11:54.257326 16157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:11:54.257340 16157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:11:54.258955 16157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.177.20:40799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.177.62:37989,127.22.177.61:38963,127.22.177.60:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/krb5kdc/kudu_127.22.177.60.keytab
--principal=kudu/127.22.177.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.177.60:35775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.177.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:11:54.259199 16157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:11:54.259397 16157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:11:54.262665 16066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } attempt: 2
I20260811 10:11:54.262920 16150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } attempt: 2
I20260811 10:11:54.263234 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.263068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57716)
0811 10:11:54.263139 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.263194 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:11:54.263293 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.263150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57724)
0811 10:11:54.263210 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.263245 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 10:11:54.263689 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.263589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57734)
0811 10:11:54.263629 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.263664 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 10:11:54.263700 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.263589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57730)
0811 10:11:54.263642 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.263673 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 10:11:54.263820 16150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:35775: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
W20260811 10:11:54.263855 16066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.177.60:35775: Network error: Client connection negotiation failed: client connection to 127.22.177.60:35775: connect: Connection refused (error 111)
W20260811 10:11:54.265450 16166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:54.265439 16163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:11:54.265442 16164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:11:54.265741 16157 server_base.cc:1034] running on GCE node
I20260811 10:11:54.266057 16157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:11:54.266407 16157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:11:54.267640 16157 hybrid_clock.cc:648] HybridClock initialized: now 1786443114267617 us; error 36 us; skew 500 ppm
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:11:54.271018 16157 init.cc:377] Logged in from keytab as kudu/127.22.177.60@KRBTEST.COM (short username kudu)
I20260811 10:11:54.271625 16157 webserver.cc:460] Webserver started at http://127.22.177.60:35097/ using document root <none> and password file <none>
I20260811 10:11:54.271901 16157 fs_manager.cc:359] Metadata directory not provided
I20260811 10:11:54.271979 16157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:11:54.272121 16157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:11:54.273299 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data/instance:
uuid: "404037b66f2d4a87ad069ce927e681b1"
format_stamp: "Formatted at 2026-08-11 10:11:54 on dist-test-slave-g350"
server_key: "8c266cc620b593bd029301e54ca6f8bf"
server_key_iv: "207e27c2fc4250c61f5a1d21e101c0e0"
server_key_version: "encryptionkey@0"
I20260811 10:11:54.273782 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal/instance:
uuid: "404037b66f2d4a87ad069ce927e681b1"
format_stamp: "Formatted at 2026-08-11 10:11:54 on dist-test-slave-g350"
server_key: "8c266cc620b593bd029301e54ca6f8bf"
server_key_iv: "207e27c2fc4250c61f5a1d21e101c0e0"
server_key_version: "encryptionkey@0"
I20260811 10:11:54.275468 16157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:11:54.276487 16173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:11:54.276700 16157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:11:54.276805 16157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data,/tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal
uuid: "404037b66f2d4a87ad069ce927e681b1"
format_stamp: "Formatted at 2026-08-11 10:11:54 on dist-test-slave-g350"
server_key: "8c266cc620b593bd029301e54ca6f8bf"
server_key_iv: "207e27c2fc4250c61f5a1d21e101c0e0"
server_key_version: "encryptionkey@0"
I20260811 10:11:54.276896 16157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:11:54.295341 16157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:11:54.297950 16157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:11:54.298127 16157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:11:54.302448 16157 rpc_server.cc:307] RPC server started. Bound to: 127.22.177.60:35775
I20260811 10:11:54.302486 16232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.177.60:35775 every 8 connection(s)
I20260811 10:11:54.302860 16157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/data/info.pb
I20260811 10:11:54.303932 16233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:11:54.305547 16233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:11:54.305882 23236 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wzoa9/build/release/bin/kudu as pid 16157
I20260811 10:11:54.306056 23236 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wzoa9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786442825609099-23236-0/minicluster-data/master-2/wal/instance
I20260811 10:11:54.306375 23236 external_mini_cluster.cc:1428] Setting key a60c46ec0a9fb99728b92bcf668cd295
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:11:54.318015 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.306457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.22.177.62:37989)
0811 10:11:54.306544 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.306547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.306761 (+ 214us) server_negotiation.cc:407] Connection header received
0811 10:11:54.307438 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.307439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.307452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.307511 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.308329 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.309251 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.310550 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.310803 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.310886 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.313745 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.313747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.313747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.313758 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.315937 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.316610 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.316612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.316612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.316782 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.317086 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.317087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.317087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.317286 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.317370 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.317796 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.317907 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 10:11:54.318568 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.306406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.62:37989 (local address 127.0.0.1:52666)
0811 10:11:54.306616 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.306661 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.307339 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.307545 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.307550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.307967 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.308209 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.308215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.309393 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.309394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.310446 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.310449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.310921 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.310922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.311046 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.311645 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.311656 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.313585 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.316031 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.316037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.316039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.316409 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.316878 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.316878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.316879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.316993 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.317411 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.317421 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.317597 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.317745 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.317940 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":3,"thread_start_us":122,"threads_started":1}
I20260811 10:11:54.318571 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.306838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.22.177.60:35775)
0811 10:11:54.307013 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.307018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.307061 (+ 43us) server_negotiation.cc:407] Connection header received
0811 10:11:54.307520 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.307534 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.307566 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.307661 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.308421 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.309754 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.310753 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.311154 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.311217 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.314434 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.314443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.314443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.314482 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.316434 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.317098 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.317101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.317101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.317147 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.317411 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.317412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.317413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.317658 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.317840 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.317952 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.318018 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260811 10:11:54.319376 16233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.62@KRBTEST.COM
I20260811 10:11:54.327483 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.319669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.61:38963 (local address 127.0.0.1:55594)
0811 10:11:54.319746 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.319769 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.319878 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.320003 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.320004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.320148 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.320187 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.320188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.320828 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.320829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.321324 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.321326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.321806 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.321807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.321871 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.322355 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.322361 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.323689 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.326092 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.326094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.326094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.326452 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.326694 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.326695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.326695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.326775 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.327098 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.327099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.327156 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.327233 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.327353 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 10:11:54.327536 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.319839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.22.177.61:38963)
0811 10:11:54.319882 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.319884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.319892 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:11:54.319944 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.319944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.319953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.320002 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.320271 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.320745 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.321418 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.321730 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.321809 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.323821 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.323822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.323823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.323829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.325909 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.326551 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.326552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.326552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.326616 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.326849 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.326850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.326851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.327007 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.327069 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.327274 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.327447 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:11:54.327874 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.319573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.22.177.62:37989)
0811 10:11:54.319619 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.319621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.319660 (+ 39us) server_negotiation.cc:407] Connection header received
0811 10:11:54.319762 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.319763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.319774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.319814 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.320399 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.320925 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.321796 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.322013 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.322098 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.324555 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.324557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.324558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.324568 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.326375 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.326859 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.326860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.326861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.326914 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.327242 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.327243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.327243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.327386 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.327583 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.327672 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.327758 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:11:54.328287 16233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.60@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.177.61@KRBTEST.COM
I20260811 10:11:54.336323 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.328683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.22.177.60:35775)
0811 10:11:54.328718 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.328719 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.328738 (+ 19us) server_negotiation.cc:407] Connection header received
0811 10:11:54.328758 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.328758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.328768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.328812 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.329078 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.329746 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.330620 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.330953 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.331041 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.333110 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.333111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.333111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.333118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.334905 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.335302 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.335303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.335303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.335368 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.335658 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.335659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.335659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.335816 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.335928 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.336091 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.336210 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260811 10:11:54.336650 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.328600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57744)
0811 10:11:54.328640 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.328653 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.328695 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.328828 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.328828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.328964 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.328999 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.329000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.329837 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.329839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.330533 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.330535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.331212 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.331213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.331273 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.331770 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.331774 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.333003 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.334998 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.334999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.335000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.335225 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.335516 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.335517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.335517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.335579 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.335952 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.335953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.335993 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.336067 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.336268 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":102}
I20260811 10:11:54.337080 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.328606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55598 (local address 127.22.177.61:38963)
0811 10:11:54.328645 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.328647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.328656 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:11:54.328677 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.328678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.328703 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.328795 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.329510 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.330279 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.331216 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.331507 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.331560 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.334085 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.334087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.334087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.334095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.335798 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.336187 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.336188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.336189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.336249 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.336524 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.336525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.336525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.336637 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.336849 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.336901 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.336980 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:11:54.338608 16233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: Bootstrap starting.
I20260811 10:11:54.338891 23236 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:11:54.339784 16233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:54.340092 16233 log.cc:824] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:54.340945 16233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: No bootstrap required, opened a new log
I20260811 10:11:54.339986 16037 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "15ad49ccfeb44beb9c18588b83618d41"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52674
I20260811 10:11:54.342585 16233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.342715 16233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404037b66f2d4a87ad069ce927e681b1, State: Initialized, Role: FOLLOWER
I20260811 10:11:54.342806 16233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.343152 16233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:54.343149 16239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.343314 16239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:11:54.345121 16251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:54.345180 16251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:54.345530 16157 master_runner.cc:186] Error getting master registration for 127.22.177.62:37989: OK, instance_id { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" instance_seqno: 1786443113914787 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:11:54.348649 16157 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:11:54.355018 16150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } attempt: 3
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443114, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.61@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:11:54.362979 16066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } attempt: 3
I20260811 10:11:54.364171 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.355542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.22.177.60:35775)
0811 10:11:54.355603 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.355605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.355617 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:54.355646 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.355646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.355658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.355734 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.356161 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.356917 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.358053 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.358356 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.358426 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.361165 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.361167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.361167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.361175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.362742 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.363200 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.363201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.363202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.363282 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.363514 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.363515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.363515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.363698 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.363756 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.363994 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.364078 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:11:54.364208 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.355446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57760)
0811 10:11:54.355516 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.355530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.355577 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.355765 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.355766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.356002 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.356067 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.356070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.357018 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.357019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.357957 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.357960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.358449 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.358451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.358520 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.359247 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.359253 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.361058 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.362831 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.362832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.362833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.363081 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.363375 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.363376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.363377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.363454 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.363765 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.363766 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.363836 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.363963 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.364094 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 10:11:54.366256 16150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: Bootstrap starting.
I20260811 10:11:54.367064 16150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:54.367447 16150 log.cc:824] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:54.368232 16150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: No bootstrap required, opened a new log
Aug 11 10:11:54 dist-test-slave-g350 krb5kdc[15980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443113, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.177.62@KRBTEST.COM for kudu/127.22.177.60@KRBTEST.COM
I20260811 10:11:54.370527 16150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.370699 16150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ab76135ad44857ad281e03425a5048, State: Initialized, Role: FOLLOWER
I20260811 10:11:54.370854 16150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.371317 16150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:54.371269 16255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.371460 16255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:54.373245 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.363308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.177.60:35775 (local address 127.0.0.1:57776)
0811 10:11:54.363371 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 10:11:54.363382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:11:54.363423 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:11:54.363945 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:11:54.363946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:11:54.364123 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:11:54.364174 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.364176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.365583 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.365584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.366551 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:11:54.366554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.367090 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.367091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:11:54.367155 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.367646 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:11:54.367652 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:11:54.369258 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:11:54.371683 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.371684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.371685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.372155 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.372432 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:11:54.372433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:11:54.372433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:11:54.372511 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:11:54.372883 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:11:54.372884 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:11:54.372949 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 10:11:54.373018 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 10:11:54.373132 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 10:11:54.373687 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.363377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.22.177.60:35775)
0811 10:11:54.363816 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.363817 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.363822 (+ 5us) server_negotiation.cc:407] Connection header received
0811 10:11:54.363836 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.363836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.363845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.363921 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.364716 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.365481 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.366656 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.366993 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.367084 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.369366 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.369367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.369367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.369375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.371560 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.372256 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.372258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.372259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.372341 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.372596 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.372597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.372598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.372768 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.372837 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.373423 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.373516 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":152,"threads_started":1}
W20260811 10:11:54.373965 16266 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:54.374025 16266 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:54.373663 16074 master_runner.cc:186] Error getting master registration for 127.22.177.62:37989: OK, instance_id { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" instance_seqno: 1786443113914787 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:11:54.375864 16074 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:11:54.377316 16066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: Bootstrap starting.
I20260811 10:11:54.378129 16066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: Neither blocks nor log segments found. Creating new log.
I20260811 10:11:54.378490 16066 log.cc:824] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: Log is configured to *not* fsync() on all Append() calls
I20260811 10:11:54.379611 16066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: No bootstrap required, opened a new log
I20260811 10:11:54.381067 16066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.381220 16066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ad49ccfeb44beb9c18588b83618d41, State: Initialized, Role: FOLLOWER
I20260811 10:11:54.381326 16066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.381692 16066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:11:54.381739 16037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 10:11:54.381816 16037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:54.382201 16269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.382385 16269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:54.382938 16037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.383211 16037 leader_election.cc:288] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [CANDIDATE]: Term 1 election: Requested vote from peers 89ab76135ad44857ad281e03425a5048 (127.22.177.61:38963), 404037b66f2d4a87ad069ce927e681b1 (127.22.177.60:35775)
I20260811 10:11:54.383608 16202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15ad49ccfeb44beb9c18588b83618d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "404037b66f2d4a87ad069ce927e681b1"
I20260811 10:11:54.383709 16124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15ad49ccfeb44beb9c18588b83618d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "89ab76135ad44857ad281e03425a5048"
I20260811 10:11:54.383769 16202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:54.383831 16124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:11:54.384696 16202 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15ad49ccfeb44beb9c18588b83618d41 in term 1.
I20260811 10:11:54.384869 16124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15ad49ccfeb44beb9c18588b83618d41 in term 1.
I20260811 10:11:54.384953 16010 leader_election.cc:302] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15ad49ccfeb44beb9c18588b83618d41, 404037b66f2d4a87ad069ce927e681b1; no voters:
I20260811 10:11:54.386468 16269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:11:54.386703 16269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [term 1 LEADER]: Becoming Leader. State: Replica: 15ad49ccfeb44beb9c18588b83618d41, State: Running, Role: LEADER
I20260811 10:11:54.386794 16269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } }
I20260811 10:11:54.387244 16269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ad49ccfeb44beb9c18588b83618d41. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.387450 16269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: This master's current role is: LEADER
W20260811 10:11:54.388398 16286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:11:54.388453 16286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:11:54.389989 16290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:11:54.390177 16290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:11:54.398924 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.386378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.22.177.62:37989)
0811 10:11:54.386437 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.386439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.389536 (+ 3097us) server_negotiation.cc:407] Connection header received
0811 10:11:54.389685 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.389686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.389698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.389733 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.390757 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.391589 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.392778 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.393097 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.393144 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.395432 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.395434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.395435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.395443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.397393 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.397895 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.397896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.397896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.397964 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.398219 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.398220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.398221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.398390 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.398484 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.398742 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.398826 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:11:54.406689 16202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [term 1 FOLLOWER]: Refusing update from remote peer 15ad49ccfeb44beb9c18588b83618d41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:54.407069 16288 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:54.408582 16239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ad49ccfeb44beb9c18588b83618d41. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.408675 16239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:54.408818 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.389270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.22.177.60:35775)
0811 10:11:54.389321 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.389323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.397765 (+ 8442us) server_negotiation.cc:407] Connection header received
0811 10:11:54.397893 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.397894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.397910 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.397946 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.398669 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.399478 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.400629 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.400964 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.401010 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.403286 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.403288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.403289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.403297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.405325 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.405812 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.405813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.405813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.405896 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.406111 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.406112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.406113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.406267 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.406396 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.406614 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.406705 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:11:54.409320 16124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [term 1 FOLLOWER]: Refusing update from remote peer 15ad49ccfeb44beb9c18588b83618d41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:11:54.409768 16288 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:11:54.411163 16255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ad49ccfeb44beb9c18588b83618d41. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.411274 16255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:11:54.411978 16014 tablet.cc:2369] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:11:54.416360 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:54.386899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55604 (local address 127.22.177.61:38963)
0811 10:11:54.387071 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.387074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.400001 (+ 12927us) server_negotiation.cc:407] Connection header received
0811 10:11:54.400112 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.400113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.400124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.400155 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.400993 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.402130 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.408319 (+ 6189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:54.408660 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:54.408705 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:54.413100 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:11:54.413101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:11:54.413102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:11:54.413110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:11:54.414969 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.415416 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.415417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.415417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.415474 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:11:54.415675 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:11:54.415676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:11:54.415676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:11:54.415817 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:11:54.415921 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:54.416158 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 10:11:54.416259 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260811 10:11:54.418579 16288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.418681 16288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:54.418779 16288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.418835 16288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41 [sys.catalog]: This master's current role is: LEADER
I20260811 10:11:54.418905 16294 mvcc.cc:205] Tried to move back new op lower bound from 7317270996565684224 to 7317270996528992256. Current Snapshot: MvccSnapshot[applied={T|T < 7317270996565684224}]
I20260811 10:11:54.419589 16239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.419696 16239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:54.420302 16255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15ad49ccfeb44beb9c18588b83618d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ad49ccfeb44beb9c18588b83618d41" member_type: VOTER last_known_addr { host: "127.22.177.62" port: 37989 } } peers { permanent_uuid: "89ab76135ad44857ad281e03425a5048" member_type: VOTER last_known_addr { host: "127.22.177.61" port: 38963 } } peers { permanent_uuid: "404037b66f2d4a87ad069ce927e681b1" member_type: VOTER last_known_addr { host: "127.22.177.60" port: 35775 } } }
I20260811 10:11:54.420398 16255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:11:54.421258 16298 mvcc.cc:205] Tried to move back new op lower bound from 7317270996565684224 to 7317270996528992256. Current Snapshot: MvccSnapshot[applied={T|T < 7317270996565684224}]
I20260811 10:11:54.421994 16290 catalog_manager.cc:1347] Generated new cluster ID: ef62f352e0154f43b8ae559495b79599
I20260811 10:11:54.422055 16290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:11:54.442726 16290 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:11:54.443679 16290 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:11:54.451308 16290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ad49ccfeb44beb9c18588b83618d41: Generated new TSK 0
I20260811 10:11:54.451534 16290 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:11:54.481483 16070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.479521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.22.177.62:37989)
0811 10:11:54.479568 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.479571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.480682 (+ 1111us) server_negotiation.cc:407] Connection header received
0811 10:11:54.480795 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.480797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.480808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.480840 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.481407 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52688: BlockingRecv error: recv got EOF from 127.0.0.1:52688 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 10:11:54.482547 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.479901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.22.177.61:38963)
0811 10:11:54.479942 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.479944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.481819 (+ 1875us) server_negotiation.cc:407] Connection header received
0811 10:11:54.481915 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.481916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.481927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.481955 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.482481 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55618: BlockingRecv error: recv got EOF from 127.0.0.1:55618 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 10:11:54.483208 16253 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.480239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57800 (local address 127.22.177.60:35775)
0811 10:11:54.480304 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.480306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.482231 (+ 1925us) server_negotiation.cc:407] Connection header received
0811 10:11:54.482299 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.482301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.482312 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.482364 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.483146 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57800: BlockingRecv error: recv got EOF from 127.0.0.1:57800 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 10:11:54.485467 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.484694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.22.177.61:38963)
0811 10:11:54.484731 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.484734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.484940 (+ 206us) server_negotiation.cc:407] Connection header received
0811 10:11:54.485041 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.485041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.485073 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.485103 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.485425 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55624: BlockingRecv error: recv got EOF from 127.0.0.1:55624 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 10:11:54.488513 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.487724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.22.177.61:38963)
0811 10:11:54.487757 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.487759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.488008 (+ 249us) server_negotiation.cc:407] Connection header received
0811 10:11:54.488103 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.488104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.488113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.488142 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.488469 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55628: BlockingRecv error: recv got EOF from 127.0.0.1:55628 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 10:11:54.492448 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.491651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.22.177.61:38963)
0811 10:11:54.491686 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.491688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.491939 (+ 251us) server_negotiation.cc:407] Connection header received
0811 10:11:54.492033 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.492034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.492043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.492072 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.492404 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55630: BlockingRecv error: recv got EOF from 127.0.0.1:55630 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 10:11:54.498502 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.497566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.22.177.61:38963)
0811 10:11:54.497671 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.497673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.497950 (+ 277us) server_negotiation.cc:407] Connection header received
0811 10:11:54.498050 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.498051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.498061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.498091 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.498452 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55638: BlockingRecv error: recv got EOF from 127.0.0.1:55638 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 10:11:54.508703 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.507718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.22.177.61:38963)
0811 10:11:54.507771 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.507773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.508091 (+ 318us) server_negotiation.cc:407] Connection header received
0811 10:11:54.508200 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.508201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.508213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.508244 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.508643 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55644: BlockingRecv error: recv got EOF from 127.0.0.1:55644 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 10:11:54.527408 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.526219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.22.177.61:38963)
0811 10:11:54.526284 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.526287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.526623 (+ 336us) server_negotiation.cc:407] Connection header received
0811 10:11:54.526757 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.526758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.526772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.526807 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.527322 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55646: BlockingRecv error: recv got EOF from 127.0.0.1:55646 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 10:11:54.561856 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.560970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.22.177.61:38963)
0811 10:11:54.561024 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.561026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.561172 (+ 146us) server_negotiation.cc:407] Connection header received
0811 10:11:54.561259 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.561260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.561274 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.561331 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.561780 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55650: BlockingRecv error: recv got EOF from 127.0.0.1:55650 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 10:11:54.627887 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.627156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.22.177.61:38963)
0811 10:11:54.627262 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.627264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.627276 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:54.627361 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.627362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.627373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.627429 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.627825 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55664: BlockingRecv error: recv got EOF from 127.0.0.1:55664 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 10:11:54.758308 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:54.757256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.22.177.61:38963)
0811 10:11:54.757356 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:54.757360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:54.757455 (+ 95us) server_negotiation.cc:407] Connection header received
0811 10:11:54.757485 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:54.757486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:54.757497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:54.757565 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:54.758106 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55674: BlockingRecv error: recv got EOF from 127.0.0.1:55674 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 10:11:55.016428 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:11:55.015676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.22.177.61:38963)
0811 10:11:55.015767 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:55.015769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:55.015781 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:55.015865 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:55.015866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:55.015887 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:55.015954 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:11:55.016363 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55680: BlockingRecv error: recv got EOF from 127.0.0.1:55680 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 10:11:55.345723 16251 catalog_manager.cc:1259] Loaded cluster ID: ef62f352e0154f43b8ae559495b79599
I20260811 10:11:55.345827 16251 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: loading cluster ID for follower catalog manager: success
I20260811 10:11:55.347299 16251 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: acquiring CA information for follower catalog manager: success
I20260811 10:11:55.347658 16251 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 404037b66f2d4a87ad069ce927e681b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:11:55.374514 16266 catalog_manager.cc:1259] Loaded cluster ID: ef62f352e0154f43b8ae559495b79599
I20260811 10:11:55.374601 16266 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: loading cluster ID for follower catalog manager: success
I20260811 10:11:55.375501 16266 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: acquiring CA information for follower catalog manager: success
I20260811 10:11:55.375854 16266 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89ab76135ad44857ad281e03425a5048: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:11:55.532858 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:55.529960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.22.177.61:38963)
0811 10:11:55.530100 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:55.530102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:55.530114 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:11:55.530165 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:55.530166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:55.530224 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:55.530317 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:55.530512 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.531046 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.531876 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.532089 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.532156 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:55.532445 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:55.532562 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:55.532625 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:55.532685 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 10:11:55.532706 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260811 10:11:55.537917 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:55.534993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.22.177.60:35775)
0811 10:11:55.535150 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:55.535151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:55.535191 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:11:55.535339 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:55.535340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:55.535379 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:55.535452 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:55.535613 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.536105 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.536988 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.537184 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.537243 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:55.537478 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:55.537587 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:55.537736 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:55.537778 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 10:11:55.537798 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260811 10:11:55.542867 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:55.539971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.22.177.61:38963)
0811 10:11:55.540020 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:55.540022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:55.540133 (+ 111us) server_negotiation.cc:407] Connection header received
0811 10:11:55.540242 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:55.540243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:55.540254 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:55.540313 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:55.540502 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.541215 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.542076 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.542325 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.542412 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:55.542599 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:55.542641 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:55.542695 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:55.542757 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 10:11:55.542781 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:11:55.543314 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:55.539830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.22.177.60:35775)
0811 10:11:55.539875 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:55.539877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:55.540090 (+ 213us) server_negotiation.cc:407] Connection header received
0811 10:11:55.540150 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:55.540151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:55.540176 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:55.540222 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:55.540436 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.541179 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.542388 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.542687 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.542746 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:55.542960 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:55.542994 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:55.543053 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:55.543196 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 10:11:55.543223 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:11:55.543489 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:11:55.539797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.22.177.62:37989)
0811 10:11:55.539965 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 10:11:55.539968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:11:55.539978 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:11:55.540031 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:11:55.540032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:11:55.540098 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:11:55.540189 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:11:55.540320 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.540913 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.542030 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:11:55.542405 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:11:55.542472 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:11:55.542719 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:11:55.542858 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:11:55.542928 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:11:55.543253 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 10:11:55.543286 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260811 10:11:55.545735 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 15993
I20260811 10:11:55.554803 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 16074
I20260811 10:11:55.563169 23236 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wzoa9/build/release/bin/kudu with pid 16157
2026-08-11T10:11:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3061 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6096 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289955 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST